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Foreword 


This  report  describes  research  conducted  as  part  of  the  Navy  Personnel  Research  and 
Development  Center's  Distributed  Training  Technology  (DTT)  project.  The  DTT  project  is  part  of 
our  Classroom  and  Afloat  Training  research  program  and  falls  under  the  Education  and  Training 
project  (L1772)  of  the  Navy’s  Manpower,  Personnel,  and  Training  Advanced  Development 
Program  Element  (0603707N).  The  work  was  performed  under  the  sponsorship  of  the  Bureau  of 
Naval  Personnel.  The  research  is  evaluating  technologies,  training  strategies,  procedures,  and 
management  methods  to  extend  videoteletraining  (VTT)  beyond  traditional,  lecture-based  courses. 

The  research  investigated  the  feasibility  of  using  videoteletraining  to  deliver  Navy  leadership 
training.  The  findings  have  direct  implications  for  the  design  of  future  distance  education  systems 
in  the  Navy  and  elsewhere. 

The  recommendations  in  this  report  are  intended  for  use  by  the  Chief  of  Naval  Education  and 
Training  and  Bureau  of  Naval  Personnel  in  developing  policy  for  the  application  of  VTT  in  the 
Navy. 
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Summary 


Problem  and  Background 

A  requirement  exists  to  train  Navy  personnel  who  are  geographically  remote  from  training 
resources.  Videoteletraining  (VTT)  enables  an  instructor  to  teach  multiple  classes  at  different 
geographic  locations.  VTT  has  been  shown  to  be  an  efficient  and  cost  beneficial  way  to  deliver 
training,  and  is  now  in  operational  use  within  the  Navy's  CNET  Electronic  Schoolhouse  Network. 
VTT  has  generally  been  used  for  the  delivery  of  lecture-based  instruction.  Navy  leadership 
training  (NAVLEAD)  involves  high  levels  of  inteipersonal  interaction  and  represents  a  departure 
from  the  instructor-centered  courses  usually  given  by  VTT.  Given  the  continuing  strong  demand 
for  NAVLEAD  training,  significant  travel  or  instructor  costs  could  be  avoided  if  such  training 
could  be  delivered  via  VTT  rather  than  in  traditional  classrooms. 

Objective 

The  objective  of  the  research  was  to  test  the  feasibility  of  using  videoteletraining  to  deliver 
NAVLEAD  training  for  the  Division  Officer's  Basic  Leadership  course. 

Approach 

Four  Division  Officer  NAVLEAD  classes  were  conducted  with  a  total  of  105  students.  Three 
treatment  groups  were  compared:  (1)  traditional  live  instruction,  (2)  VTT  local,  and  (3)  VTT 
remote.  Two  classes  were  given  in  the  traditional  manner  with  students  and  instructors  in  a 
conventional  classroom.  Two  other  classes  were  given  by  VTT.  A  total  of  36  students  were  in 
VTT  local  classrooms  with  an  instructor,  and  another  22  students  were  in  remote  VTT  classrooms 
connected  to  the  local  classroom  by  a  two-way  audio  and  video  videoteletraining  system. 

The  treatment  groups  were  compared  in  terms  of  five  outcome  measures  reflecting  student 
evaluations  of  VTT,  student  evaluations  on  instructional  topics,  daily  observer  evaluations  on 
several  dimensions  of  the  training,  class  participation,  and  student  knowledge  gained. 

Results  and  Conclusions 

Student  responses  on  questionnaires  tended  to  favor  traditional  instruction  slightly,  but 
differences  were  small.  The  few  differences  between  groups  on  the  VTT  questionnaire  related  to 
audiovisual  factors  and  a  tendency  for  remote  students  to  express  the  opinion  that  VTT  reduced 
their  opportunities  to  interact  with  the  instructor  and  other  students.  Although  a  third  of  the 
students  preferred  traditional  instruction,  nearly  nine  out  of  ten  were  willing  to  take  another  VTT 
course.  On  the  questionnaire  covering  instructional  issues,  the  largest  differences  between  groups 
in  favor  of  traditional  instruction  were  primarily  on  topics  related  to  seeing  and  hearing  students, 
teams,  and  instructors  or  on  topics  related  to  interaction  and  participation. 

Subject  matter  expert  ratings  of  perceived  effectiveness  of  different  aspects  of  the  course  were 
significantly  higher  for  traditional  instruction  than  VTT.  Though  improvement  occurred 
throughout  the  week,  VTT  courses  did  not  reach  parity  with  traditional  courses. 
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A  class  participation  tally  indicated  a  higher  level  of  student  initiated  questions  and  comments 
in  traditional  than  in  VTT  classes.  Some  variability  was  observed  in  that  one  traditional  class 
interacted  at  twice  the  level  of  the  other,  which  was  in  turn  more  similar  to  the  level  in  VTT  local 
classes.  VTT  remote  classes  interacted  at  about  two  thirds  the  level  of  VTT  local  classes. 

A  test  on  course  content  showed  virtually  identical  levels  of  knowledge  among  traditional, 
VTT  local,  and  VTT  remote  students. 

The  present  evaluation  showed  that  it  was  possible  to  deliver  NAVLEAD  by  VTT,  given  some 
reduction  in  participation  and  interaction.  The  classes  studied  were  a  first  attempt  at  delivering  this 
instruction  by  VTT.  If  the  course  were  delivered  by  VTT  on  a  regular  basis,  it  is  possible  that 
instructors  would  further  develop  techniques  to  foster  interaction.  VTT  delivery  of  the  course 
would  yield  cost  savings  associated  with  travel  and  instructors.  The  generality  of  these  findings 
will  be  examined  in  a  subsequent  report  on  the  feasibility  of  using  VTT  for  Leading  Petty  Officer 
(LPO)  and  Chief  Petty  Officer  (CPO)  NAVLEAD  courses.  That  evaluation  includes  an  additional 
measure  of  student  performance  on  a  simulated  activity  that  could  not  be  collected  in  the  present 
evaluation. 

Recommendations 

1.  In  making  its  decision  to  use  VTT  for  NAVLEAD,  the  Chief  of  Naval  Education  and 
Training  should  consider  both  this  evaluation  and  that  subsequently  conducted  for  Chief  and 
Leading  Petty  Officer  courses. 

2.  If  the  decision  is  made  to  conduct  NAVLEAD  instruction  with  VTT,  the  Chief  of  Naval 
Education  and  Training  should  test  and  refine  ways  to  foster  higher  levels  of  instructor-student 
and  student-student  interaction. 
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Introduction 


Problem  and  Background 

A  requirement  exists  to  train  Navy  personnel  who  are  geographically  remote  from  training 
resources.  An  increasingly  efficient  approach  to  meeting  this  requirement  is  needed  as  the  Navy 
downsizes  and  training  resources  become  constrained.  Previous  research  and  development  work 
has  demonstrated  that  videoteletraining  (VTT)  can  be  an  efficient  method  to  deliver  training 
electronically  to  remote  Navy  personnel  (Rupinski  &  Stoloff,  1990;  Rupinski,  1991;  Simpson, 
Pugh,  &  Parchman,  1990, 1991, 1992;  Stoloff,  1991).  The  Chief  of  Naval  Education  and  Training 
(CNET)  now  has  VTT  in  operational  use  in  the  CNET  Electronic  Schoolhouse  Network  (CESN). 
To  date,  most  VTT  has  been  limited  to  the  delivery  of  lecture-based  training.  Present  VTT 
technology  reduces  the  quality  of  the  audio  and  video  as  compared  to  live  instruction;  e.g.,  it 
reduces  the  visibility  of  personnel  at  different  classroom  locations  and  also  reduces  the  ability  of 
instructors  and  students  to  interact  as  in  a  traditional  classroom.  These  constraints  make  it  difficult 
to  conduct  training  which  is  not  instructor  centered  and  which  involves  high  levels  of  student 
participation  such  as  Navy  leadership  training  (NAVLEAD).  Given  the  continuing  strong  demand 
for  NAVLEAD  training,  significant  costs  could  be  avoided  if  such  training  could  be  delivered  via 
VTT  rather  than  in  traditional  classrooms.  Moreover,  success  with  NAVLEAD  would  open  the 
possibility  of  using  VTT  to  conduct  training  in  total  quality  leadership,  core  values,  and  other 
subjects  involving  highly-interactive  training. 

Objective 

The  objective  of  the  Distributed  Training  Technology  (DlT)  project  is  to  evaluate 
technologies,  training  strategies,  procedures,  and  management  methods  to  extend  VTT  beyond 
traditional  lecture-based  courses.  This  report  describes  research  conducted  to  test  the  feasibility  of 
delivering  NAVLEAD  training  via  VTT. 

Research  Issues 

NAVLEAD  training  poses  unique  challenges  to  VTT  because  it  differs  from  traditional 
lecture-based  instruction  in  several  ways.  Among  other  things,  there  is  a  serious  attempt  to  impart 
attitudes  and  values  in  this  training  as  well  as  factual  knowledge  and  skills.  The  training  involves 
a  combination  of  lecture,  discussion,  experiential  learning,  and  team-building  activities  which 
occur  in  a  highly-interactive  learning  environment  that  requires  students  to  make  decisions,  take 
positions,  defend  themselves  before  their  peers,  work  as  team  members,  and  take  responsibility  for 
their  own  learning.  The  intensity  of  NAVLEAD  training  is  abetted  by  having  a  team  of  instructors 
conduct  training,  making  a  determined  effort  to  draw  out  students  to  facilitate  instructor-student 
and  student-student  interaction,  and  arranging  the  classroom  so  that  instructors  are  able  to  stroll 
among  tables  in  physical  proximity  to  students.  NAVLEAD  instructors  are  called  “facilitators,”  in 
formal  acknowledgment  that  they  are  not  the  sole  or  even  primary  source  of  knowledge  but  are 
present  to  facilitate  a  learning  process  in  which  the  students  themselves  share  knowledge  and 
experiences  and  learn  from  one  another.  Facilitators  are  trained  to  interpret  nonverbal  cues  such  as 
body  language  and  facial  expressions  and  use  them  to  assess  student  understanding  and  attitudes. 
NAVLEAD  training  stresses  team  building.  Students  are  organized  in  small  groups  and  work 
together  as  a  unit  throughout  the  course.  The  team  is  assigned  group  problem-solving  tasks  and 
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members  work  together  in  establishing  roles  and  group  hierarchy,  defining  and  solving  problems, 
and  reporting  back  to  the  class.  Successful  teams  are  cohesive  and  group  members  work  effectively 
together,  frequently  taking  on  a  group  identity  that  is  competitive  with  respect  to  other  groups. 

Given  the  foregoing,  some  key  test  and  evaluation  issues  in  delivering  NAVLEAD  by  VTT  are 
the  following; 

•  Will  VTT  permit  the  highly  interactive  instructional  environment  of  the  live  classroom,  or 
will  it  be  compromised  by  lack  of  physical  proximity  of  instructor  and  students? 

•  Can  experiential  learning  activities  such  as  case  studies,  exercises,  and  simulations  be 
conducted  successfully  using  VTT? 

•  Will  VTT  affect  student  performance  or  knowledge  acquisition? 

•  Will  VTT  affect  student  attitudes  toward  the  learning  experience? 

The  research  described  in  this  report  focused  on  these  issues. 

Method 

The  method  of  the  study  is  described  below  in  terms  of  the  research  plan,  data  collection 
instruments,  and  the  preparation  of  the  course  for  VTT. 

Research  Plan 

The  research  plan,  described  below,  consists  of  research  objectives,  research  design  and 
independent  variable,  dependent  variables,  data  collection  methods  and  instruments,  subjects,  and 
data  collection. 

Research  Objectives 

The  objective  of  this  research  effort  was  to  test  the  feasibility  of  using  VTT  to  deliver 
NAVLEAD  training  for  the  Division  Officer’s  Basic  Leadership  course.  Feasibility  was  defined  in 
terms  of  several  general  criteria,  including  student  perceptions  of  VTT  and  training  quality,  student 
knowledge  gained,  facilitator/observer  perceptions  of  training  quality,  student  performance,  and 
class  participation.  The  baseline  for  comparison  was  traditional  live  instruction.  The  use  of  VTT 
technology  would  not  be  expected  to  improve  training  in  terms  of  the  general  criteria;  parity  with 
traditional  instruction  would  validate  the  use  of  VTT.  The  research  objective  was  addressed  by 
determining  the  effects  upon  dependent  variables  of  student  participation  in  traditional  live 
instruction  versus  VTT  instruction. 

Research  Design  and  Independent  Variable 

A  single  independent  variable  (type  of  instruction)  with  three  states  was  used.  This  variable 
consisted  of  three  treatment  groups;  (1)  traditional  classrooms;  (2)  VTT  local  classrooms  with 
students  and  an  instructor,  and  (3)  VTT  remote  classrooms  where  students  were  connected  to  the 
local  classroom  by  a  two-way  audio  and  video  VTT  system. 
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Dependent  Variables 

Dependent  variables  fell  into  six  general  classes:  (1)  student  perceptions  of  VTT  quality,  (2) 
student  perceptions  of  training  quality,  (3)  student  knowledge  gained,  (4)  facilitator/observer 
perceptions  of  training  quality,  (5)  student  performance,  and  (6)  class  participation.  Each  of  these 
variables  was  defined  in  terms  of  several  related  measures  which  were  gathered  with  the  six  data 
collection  instruments  described  below. 

Subjects 

Subjects  were  active  duty  Navy  officers  in  a  variety  of  career  designators  and  ranks  ranging 
from  0-1  to  0-4;  mean  rank  was  0-2.2.  The  traditional  course  was  always  given  at  Little  Creek, 
VA.  The  research  plan  called  for  20  students  at  each  of  three  sites  during  VTT  classes,  but  this  was 
not  possible  to  accomplish  in  practice.  During  VTT  classes,  the  local  classroom  (originating  site 
with  the  instructor)  was  located  at  Dam  Neck,  VA.  A  remote  classroom  was  located  at  Charleston, 
SC  during  the  first  VTT  class,  and  at  both  Charleston  and  Newport,  RI  during  the  second  VTT 
class.  Fewer  students  attended  than  had  been  planned.  Table  1  gives  the  number  of  students  and 
mean  rank  by  location  and  class  type  for  the  four  NAVLEAD  classes  included  in  the  research 
study. 


Table  1 


Class  Types,  Locations,  Number  of  Students,  and  Average  Student 
Military  Rank  by  Class  Number 


Class 

Number 

Class 

Type 

Location 

Number 
of  Students 

Average 

Mil  Rank 

1 

VTT  (local) 

Dam  Neck 

23 

0-2.5 

VTT  (remote) 

Charleston 

8 

0-2.0 

Overall 

31 

0-2.4 

2 

Traditional 

Little  Creek 

27 

0-1.8 

3 

VTT  (local) 

Dam  Neck 

13 

0-2.3 

VTT  (remote) 

Charleston 

5 

0-2.4 

VTT  (remote) 

Newport 

9 

0-2.0 

Overall 

27 

0-2.2 

4 

Traditional 

Little  Creek 

20 

0-2.1 

All 

VTT  (local) 

36 

0-2.4 

All 

VTT  (remote) 

22 

0-2.1 

All 

Traditional 

47 

0-1.9 

All 

Total 

105 

0-2.2 
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Data  Collection  Instruments 


Six  different  data  collection  instruments  were  used.  These  instruments  were  designated  Q.l 
through  Q.6.  The  purpose  and  content  of  each  instrument  are  described  below.  Table  2  indicates 
who  was  required  to  complete  each  instrument,  when  it  was  completed,  and  the  approximate  time 
requited  for  completion.  Instructions  for  administering  these  instruments  are  given  in  Appendix  A. 
Because  Q.  1  asked  questions  pertaining  to  VTT  issues,  it  was  administered  only  to  students  in  VTT 
classes  and  no  comparable  data  were  obtained  from  traditional  classes.  All  other  instruments  were 
administered  in  both  traditional  and  VTT  classes  and  provided  dependent  measures  that  enabled 
comparisons  among  the  three  treatment  groups. 

Table  2 


Data  Collection  Overview 


Course  Type 

Completed 

When 

Est  Time 

Instrument 

Traditional 

VTT 

by 

Completed 

Required 

Q.l 

No 

Yes 

VTT  Students 

Day  5 

5  min. 

Q.2 

Yes 

Yes 

All  Students 

Day  5 

10  min. 

Q.3 

Yes 

Yes 

All  Students 

Days  1  &  5 

25  min. 

Q.4 

Yes 

Yes 

Facil/Observ 

Daily 

10  minVday 

Q.5 

Yes 

Yes 

Facil/Observ 

Days  3  &  5 

5  min. 

Q.6 

Yes 

Yes 

Facil/Observ 

Daily 

2  hrs/day 

VTT  Student  Questionnaire  (Q.l):  Student  perceptions  of  VTT  quality  were  assessed  at 
the  conclusion  of  the  course  with  Q.l  (Appendix  B).  It  consists  of  ratings  of  video,  audio, 
VTT  procedures,  local  vs.  remote  team  participation;  and  multiple-choice  items  regarding 
student  preferences. 

NAVLEAD  Student  Questionnaire  (Q.2):  Student  perceptions  of  training  quality  were 
assessed  at  the  conclusion  of  the  course  with  Q.2  (Appendix  C).  This  questionnaire 
employs  items  concerned  with  ratings  of  facilitators,  personnel  visibility  and  audibility, 
written  materials,  learning  activities,  training  aids,  interaction/participation,  overall 
evaluation;  and  open-ended  questions  about  student  likes,  dislikes,  and  suggestions. 

NAVLEAD  Quiz  (Q.3):  Student  knowledge  of  course  content  was  assessed  twice  with  a 
quiz  (Q.3,  as  given  in  Appendix  D).  This  quiz  was  administered  as  a  pretest  to  students 
before  the  course  and  again  as  a  posttest  at  the  end  of  the  course.  The  items  in  this  quiz 
cover  knowledge  of  course  content,  which  was  assessed  with  25  items  given  in  a  four- 
alternative  multiple-choice  format. 

Fadlitator/Observer  Questionnaire  (Q.4):  Facilitator  or  observer  perceptions  of  training 
quality  were  assessed  on  a  daily  basis  with  Q.4  (Appendix  E).  It  consists  of  ratings  on  various 
dimensions  of  the  course  from  the  perspective  of  the  facilitator  or  subject  matter  expert 
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(SME)  (e.g.,  effectiveness  of  presentations  and  exercises,  success  in  meeting  learning 
objectives,  difficulty  of  conducting  instruction,  student  interaction,  student  participation, 
degree  of  control,  presentation  quality,  cohesiveness;  space  is  provided  for  comments). 

Performance  Activities  (Q.5):  Q.5  (Appendix  F)  reflects  student  performance  and 
consists  of  facilitator/observer  ratings  on  several  different  dimensions  of  student 
performance  in  completing  classroom  performance  activities  for  lesson  3.5  (DIVO/CPO 
relationships)  and  5.4  (assessing  the  division). 

Class  Participation  Tally  (Q.6):  Class  participation  was  assessed  with  Q.6  (Appendix  G) 
by  having  an  observer  record  a  frequency  tally  of  the  number  of  student-initiated  questions 
and  comments  from  each  student  team.  Remarks  had  to  be  directed  at  facilitators  or 
students,  had  to  relate  to  course  content,  and  had  to  be  intended  for  the  class  to  hear.  These 
interaction  tallies  were  recorded  daily  during  two  one-hour  intervals  (0900-l(XX)  and  1300- 
1400)  for  all  classes. 

Data  Collection 

All  data  collection  instruments  were  administered  by  NAVLEAD  facilitators  or  other 
designated  personnel  according  to  procedures  in  a  set  of  data  collection  instructions  (Appendix  A). 
Data  were  collected  from  facilitators,  observers,  and  students  during  four  iterations  of  the  Division 
Officer’s  NAVLEAD  course  (2  traditional  and  2  VTT).  The  ideal  data  collection  sequence  would 
have  been  traditional- VTT- VTT-traditional  or  VTT-traditional-traditional- VTT,  with  an  equal  and 
brief  time  gap  separating  each  course  iteration.  Real-world  scheduling  and  quota  constraints  led  to 
the  following  actual  sequence  of  classes  by  type  and  date: 

1.  VTT  (28  Sept  -  2  Oct  1992) 

2.  Traditional  (19-23  Oct  1992) 

3.  VTT  (26-30  Oct  1992) 

4.  Traditional  (11-15  Jan  1993) 

To  assure  comparability  of  traditional  and  VTT  data,  it  would  have  been  desirable  to  use  the 
same  facilitators  and  observers  throughout  the  four  course  iterations.  This  was  possible  in  the  case 
of  the  facilitators,  but  not  for  observers. 

Preparation  for  VTT 

Instruction  delivered  by  VTT  typically  requires  several  adaptations  of  the  training  materials 
and  classroom  configuration,  as  well  as  instructor  preparation  for  using  the  medium.  The  methods 
specific  to  the  NAVLEAD  course  are  given  below.  A  general  guide  for  converting  courses  to  VTT 
is  given  in  Simpson  (1993). 

Training  Course  Selection 

The  research  was  conducted  with  the  Division  Officer  Basic  Leadership  Course,  as  described  in 
the  course  Instructor  and  Student  guides  (HGL  Associates,  1992a,  1992b).  The  course  was  a  revised 
version  of  one  previously  taught  and  its  introduction  occurred  concurrently  with  the  research  study. 
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NAVLEAD  training  combines  lecture,  discussion,  experiential  learning,  and  team-building 
activities  which  are  team-taught  in  a  classroom  whose  physical  layout  permits  facilitators  to  stroll 
among  tables  in  physical  proximity  to  students.  The  intense,  highly-interactive  learning 
environment  encourages  students  to  share  knowledge  and  experiences,  to  learn  from  one  another, 
and  stresses  team-building.  Facilitators  rely  on  their  perception  and  interpretation  of  student 
nonverbal  cues  to  guide  the  learning  activities.  During  lectures  and  discussion,  facilitators  use 
overhead  transparencies,  videotapes,  posters,  and  various  handouts.  The  student  guide  contains 
case  studies,  exercises,  and  simulations.  No  formal  testing  is  included  in  the  course. 

The  course  differs  from  previous  courses  we  have  studied  because  of  its  high  level  of 
interactivity  (facilitator-student,  student-student),  use  of  experiential  learning  activities,  use  of 
nonverbal  information,  and  stress  on  team  building. 

Classroom  Design  Adaptations 

Students  in  traditional  NAVLEAD  classrooms  sit  with  their  teammates  around  tables,  typically 
with  six  students  per  table,  as  illustrated  in  Figure  la.  This  arrangement  enables  students  to 
communicate  easily  with  teammates  and  allows  facilitators  to  stroll  around  the  room  in  close 
proximity  to  tables  and  students.  The  VTT  classrooms  used  in  the  research  were  arranged  as 
illustrated  in  Figure  lb.  Students  were  still  assigned  to  teams,  though  team  members  had  to  rotate 
to  face  one  another  during  group  work.  In  addition,  this  arrangement  did  not  permit  facilitators  the 
same  intimacy  with  students  in  local  or  remote  classrooms. 

The  VTT  classrooms  were  equipped  with  a  fully  interactive  two-way  video  and  audio  VTT 
system.  The  local  classroom  was  equipped  with  a  remotely-controlled  instructor  camera,  a  student 
camera,  an  electronic  presentation  device,  and  a  video  easel  camera.  One  classroom  camera  was 
aimed  to  show  a  view  of  the  instructor  and  another  student  camera  was  aimed  to  display  a  view  of 
the  students.  The  camera  operator/technician  selected  cameras  or  other  video  input  devices  as  well 
as  controlling  other  aspects  of  audio  and  video  through  a  control  panel  and/or  infrared  remote 
controls.  Visual  aids  were  presented  on  a  TV  monitor.  The  video  easel  camera  was  used  to  display 
material  that  might  more  commonly  be  displayed  using  an  overhead  projector.  Transparency 
material  was  also  presented  in  computer-based  electronic  form  with  a  General  Parametrics 
Corporation  VideoShow  electronic  slide  presentation  device. 

Students  in  the  local  classroom  observed  facilitators  and  students  in  that  classroom  directly  but 
could  see  students  in  the  remote  classroom  only  on  a  TV  monitor.  Students  in  remote  classrooms 
observed  NAVLEAD  facilitators  or  students  in  the  local  classroom  on  a  TV  monitor,  depending 
upon  which  camera  was  selected. 

Training  Adaptations 

Training  was  adapted  for  VTT  through  a  working  collaboration  between  representatives  of  the 
CESN  and  the  NAVLEAD  school  at  Little  Creek.  No  modifications  were  made  to  the  content  of 
lectures  or  course  written  materials.  VTT  and  traditional  class  length  were  identical.  However,  the 
VTT  classes  differed  in  some  ways  from  traditional  classes: 
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•  Visuals  or  transparencies  were  presented  on  TV  monitors  by  using  the  video  e^l  camera 
or  by  the  electronic  slide  presentation  device  instead  of  on  conventional  projection  screens 
or  whiteboards. 

•  All  posters  were  converted  to  hardcopy  form  and  provided  to  students  instead  of  posting 
them  on  walls. 

•  Students  were  briefed  on  the  network  and  encouraged  to  become  medium  conscious,  e.g., 
by  pressing  a  button  on  the  microphone  before  speaking  so  that  they  could  be  heard  by 
students  at  other  sites. 

•  Facilitators  limited  their  range  of  body  motion  to  remain  on  camera  and  their  rate  of  motion 
to  prevent  image  jerkiness  associated  with  the  rate  of  video  transmission. 

•  Facilitators  made  special  efforts  to  maintain  participation  by  students  at  remote  sites,  for 
example,  by  soliciting  comments  and  directing  questions  at  individuals. 

•  Facilitators  used  the  video  easel  camera  vs  whiteboard  or  flip  chart  for  compiling 
classroom  comments. 

Instructor  Training 

Four  different  instructors  (facilitators)  delivered  training  during  the  research  study.  Instructors 
team-taught  the  classes.  Instructors  were  familiarized  with  the  audio  and  video  equipment  and 
practiced  equipment  operation  and  class  procedures.  The  total  training  period  per  instructor  was 
approximately  two  days,  most  of  it  devoted  to  practice  teaching.  None  of  the  instructors  were  given 
or  had  previously  received  training  in  camera  presence,  articulation,  graphics  production,  or  other 
skills  of  TV  professionals. 


Results 

Traditional,  VTT  local,  and  VTT  remote  treatment  conditions  are  compared  below  for  each  of 
the  data  collection  instruments,  Q.l  through  Q.6. 

VTT  Student  Questionnaire  (Q.l) 

Student  attitudes  were  measured  with  a  post-course  questionnaire  (Appendix  B)  which 
contained  a  series  of  statements  to  be  rated,  multiple-choice  questions,  and  open-ended  questions. 
Questionnaires  were  only  completed  by  the  58  students  participating  in  the  VTT  class  convenings 
(i.e.,  classes  number  1  and  3  as  shown  in  Table  1).  The  results  are  presented  in  terms  of 
comparisons  between  students  in  the  local  (originating)  classroom  and  those  in  the  remote 
classrooms. 

Student  Ratings 

The  statements  rated  by  students  (items  1-17  in  Appendix  B)  were  grouped  in  three  categories 
(video,  audio,  VTT  procedures).  Statements  were  rated  on  a  5-point  scale  with  a  midpoint  of  3.  A 
1  indicated  “Unsatisfactory”  and  a  5  indicated  “Outstanding.”  Mean  ratings  on  each  item  were 
computed  for  all  local  and  all  remote  classrooms  across  course  convenings.  Results  are  shown  in 
Figure  2.  Group  averages  on  each  of  the  items  are  above  the  midpoint  on  the  rating  scale;  most 
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students  gave  positive  ratings  to  the  dimension  being  measured.  Differences  between  local  and 
remote  classrooms  were  smdl  and  in  most  cases  do  not  show  interesting  patterns  and  suggest  that, 
in  the  main,  students  in  both  local  and  remote  classrooms  were  positively  disposed  toward  several 
different  aspects  of  the  VTT  learning  environment,  a  result  that  generally  parallels  previous  VTT 
research  (see  Simpson  et  al.,  1990, 1991, 1992). 

One-way  analyses  of  variance  (ANOVA)  were  conducted  to  determine  the  statistical  significance 
of  rating  differences  between  local  and  remote  classrooms.  Statistically  significant  differences  were 
obtained  for  question  2  (F(l,57)  =  4.87,  p  <  .05),  question  3  (F(l,57)  =  16.24,  p  <  .01,  and  question 
7  (F(l,57)  =  10.47,  p  <  .01).  Responses  to  these  questions  indicate  that  remote  students  had  more 
problems  with  audiovisual  factors  than  students  in  the  local  classroom. 

Multiple-Choice  Questions 

Student  perceptions  of  their  opportunities  to  interact  and  their  preference  for  a  method  of 
instruction  were  assessed  with  multiple-choice  questions  (items  18-22  in  Appendix  B).  Items  18-20 
had  three  choices  and  items  21  and  22  had  two.  Percentages  of  response  to  each  choice  were 
calculated  for  local  and  remote  classrooms,  and  Chi  Square  tests  were  used  to  compare  response 
distributions  by  classroom. 

Figure  3a  shows  student  responses  to  question  18,  which  asked  “How  did  the  VTT  method  of 
instruction  affect  your  opportunities  to  interact  with  the  instructor?”  A  Chi  Square  test  showed  that 
the  distributions  differed  significantly  (x^(2)  =  10.11,  p  <  .01).  The  majority  of  students  in  both 
classrooms  felt  that  VTT  had  no  effect  on  opportunities  to  ask  questions.  However,  compared  to 
local  students,  remote  students  were  less  likely  to  respond  that  there  was  no  effect  and  were  more 
likely  to  respond  that  VTT  provided  fewer  opportunities. 

Figure  3b  shows  student  responses  to  question  19,  which  asked  “How  did  the  VTT  method  of 
instruction  affect  your  opportunities  to  interact  with  other  students?”  A  Chi  Square  test  showed  that 
the  distributions  differed  significantly  (x^(2)  =  10.24,  p  <  .01).  The  pattern  of  responses,  local 
versus  remote,  shows  that  remote  students  were  less  favorably  disposed  toward  VTT  than  local 
students  in  terms  of  ability  to  interact  with  other  students. 

Figure  4a  shows  student  responses  to  question  20,  which  asked  “Which  method  of  instruction 
would  you  have  preferred  for  this  course?”  A  Chi  Square  test  showed  no  significant  difference  in 
the  response  distributions  although  the  response  rates  appear  to  differ  between  local  and  remote 
classrooms;  perhaps  the  most  important  difference  is  that  remote  students  were  about  half  as  likely 
as  local  to  prefer  VTT  instruction. 

Question  21  asked  “Which  of  the  following  would  you  prefer;  (a)  Enrolling  in  a  VTT  course 
near  your  home  port  or  (b)  Enrolling  in  a  traditional  (live)  course  farther  (TAD)  from  your  home 
port.”  The  majority  of  students  in  both  classrooms  preferred  taking  a  VTT  course  nearby  to  a 
traditional  course  requiring  travel  (Figure  4b). 

(Question  22  asked  “Which  of  the  following  would  you  prefer,  (a)  Enrolling  in  a  VTT  course  at  a 
time  convenient  to  your  time  schedule  or  (b)  Enrolling  in  a  traditional  (live)  course  at  time  inconvenient 
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Question 

•  Video 

1.  Image  on  TV  was  large  enough  to  be  seen 

2.  Image  on  TV  was  clear  enough  to  be  seen 

3.  Graphics/slides/transparencies  on  TV  were  readable 

•  Audio 

4.  Audio  from  other  class  was  loud  enough  to  understand 

5.  Audio  from  other  class  was  clear  enough  to  understand 

6.  Instructor’s  voice  could  be  heard  adequately 

7.  Student  voices  could  be  heard  adequately 

8.  Microphones  were  convenient  to  use 

•  VTT  Procedures 

9.  Students  knew  how  to  use  microphones 

10.  Students  knew  how  to  stay  on  camera 

11.  Students  knew  how  to  attract  instructor’s  attention 

12.  Instructors  handled  questions/comments  effectively 

13.  Instructors  coordinated  activities  among  classes  effectively 

14.  VTT  technical  problems  were  resolved  in  a  timely  manner 

15.  Instructors  were  prepared  to  teach  on  the  VTT  network 

16.  Thera  was  good  cohesiveness  between  local  and  remote  sites 

17.  Local  and  remote  sites  participated  equally  in  the  ciass 

ZZ3  VTT  Local 
■■  VTT  Remote 


Figure  2.  VTT  Student  Questionnaire  (Q.l)  responses. 
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a)  Opportunities  to  interact 
with  instructor 


b)  Opportunities  to  interact 
with  other  students 


Loca 


Remote 


Local 


Classroom 


Figure  3.  VTT  Student  Questionnaire  (Q.l)  responses  to  items  18  and  19  on  how  VTT 
affected  opportunities  to  interact  with  the  instructor  or  other  students. 

to  your  time  schedule.”  Over  nine  tenths  of  the  students  in  either  classroom  preferred  attending  a  VTT 
course  at  a  convenient  time  to  a  traditional  course  at  an  inconvenient  time  (Figure  4c). 

Comments 

Student  comments  were  solicited  with  question  23  (“If  you  had  a  choice,  would  you  take 
another  VTT  course?”),  to  which  the  student  answered  yes  or  no  and  then  explained  the  chosen 
response  by  writing  in  comments.  Figures  5a  and  5b  show  the  responses  to  this  two-part  question 
in  terms  of  type  or  response  for  local  and  remote  classrooms. 

Figure  5a  shows  responses  to  the  first  part  of  the  question.  The  majority  of  students  in  both 
local  and  remote  classrooms  answered  “yes”  to  the  question,  indicating  that  they  would  take 
another  VTT  course.  There  was  little  difference  between  local  and  remote  students. 

The  comments  made  in  the  second  part  of  question  23  were  exhaustively  listed  and  categorized 
as  positive  or  negative,  as  shown  in  Figure  5b.  A  total  of  63  comments  was  made  by  50  students 
(9  students  did  not  comment).  The  comments  are  distributed  about  evenly  between  positive  and 
negative  in  either  local  or  remote  classrooms.  This  ambivalence  may  suggest  less  enthusiasm  in 
the  willingness  to  participate  in  future  VTT  courses  than  was  shown  in  the  first  part  of  the  question 
(i.e.,  although  roughly  9  out  of  10  students  expressed  willingness  to  participate,  5  out  of  10  of  those 
who  commented  had  negative  comments  on  the  prospect). 
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Percent  Response 


a)  Preference  for  method  b)  Preference  for  c)  Preference  for 

of  instruction  convenient  iocation  convenient  time 


Local  Remote  Locai  Remote  Local  Remote 


Classroom 

Figure  4.  Student  preferences  for  method  of  instruction,  location,  and  time  on  items  20, 21, 
and  22  of  VTT  Student  Questionnaire  (Q.l) 
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NAVLEAD  Student  Questionnaire  (Q.2) 

Student  attitudes  were  measured  with  a  post-course  questionnaire  (Appendix  C)  which  contained 
27  statements  to  be  rated  and  three  open-ended  questions.  Questionnaires  were  completed  by  all  105 
students  participating  in  both  traditional  and  VTT  classes  (i.e.,  classes  number  1  through  4  as  shown 
in  Table  1).  There  is  some  similarity  between  the  VTT  Student  Questionnaire  (Q.l)  and  the 
NAVLEAD  Student  Questionnaire  (Q.2).  Q.l  is  the  more  limited  instrument,  as  it  focuses  on 
questions  that  could  only  be  answered  by  students  participating  in  VTT  classes  and  the  only  statistical 
comparisons  it  permits  are  between  VTT  local  and  V’lT  remote.  Q.2  has  broader  coverage,  can  be 
administered  to  students  regardless  of  whether  VTT  is  involved,  and  permits  comparisons  between 
VTT  and  traditional  classes.  The  analyses  described  below  compared  NAVLEAD  as  delivered  in 
three  different  forms:  (1)  traditional,  (2)  VTT  local,  (3)  VTT  remote. 

Student  Ratings 

The  statements  rated  by  students  (items  1-27  in  Appendix  C)  were  grouped  in  six  categories 
(instructor  performance,  students/team,  learning  activities,  training  aids,  interaction/participation, 
and  overall  course  rating).  As  with  Q.l,  statements  were  rated  on  a  5-point  scale  with  3  as  a 
midpoint,  1  indicating  “Unsatisfactory,  and  5  indicating  “Outstanding.”  Mean  ratings  were 
computed  for  traditional,  VTT  local,  and  VTT  remote.  Results  are  shown  in  Figure  6.  Group 
averages  on  each  of  the  items  were  above  the  midpoint  on  the  rating  scale;  most  students  gave 
positive  ratings  to  the  dimension  being  measured.  Comparison  of  the  ratings  indicates  that  highest 
ratings  were  given  for  traditional,  slightly  lower  for  VTT  local,  and  lowest  for  VTT  remote;  mean 
ratings  over  items  1-27  in  these  three  groups  are  traditional  (4.6),  VTT  local  (4.5),  and  VTT  remote 
(4.2).  While  this  is  the  overall  trend,  rating  differences  vary  by  item  from  negligible  (e.g.,  items  4, 
14, 15)  to  half  a  rating  point  or  more  (e.g.,  items  7, 8, 10, 12, 23).). 

Analyses  of  variance  (ANOVA)  were  conducted  to  determine  the  statistical  significance  of 
rating  differences  among  traditional,  VTT  local,  and  VTT  remote  classrooms.  The  results  for  the 
main  effect  tests  are  shown  in  the  ANOVA  T-L-R  columns  of  Table  3.  In  addition,  Tukey  HSD 
tests  were  conducted  to  determine  whether  any  of  the  pairwise  combinations  of  the  three  groups 
exceeded  a  difference  threshold  at  the  .05  level  of  significance.  Pairwise  combinations  are 
differences  among  traditional  (T),  VTT  local  (L),  and  VTT  remote  (R),  i.e.,  T-L,  T-R,  L-R.  If  a 
main  effect  is  significant,  the  HSD  tests  allow  an  examination  of  the  pairwise  difference 
combinations  that  contribute  to  that  main  effect.  The  result  of  these  tests  are  shown  by  asterisks 
next  to  the  mean  differences  given  in  the  last  three  HSD  columns  of  Table  3. 

Significant  main  effects  are  absent  within  all  blocks  of  items  relating  to  learning  activities 
(items  14-17),  training  aids  (18-20),  and  overall  (25-27).  These  findings  suggest  that  regardless  of 
what  type  of  classroom  the  student  was  in,  the  attitude  expressed  was  about  the  same  toward  case 
studies,  exercises,  simulations,  written  materials,  training  aids,  instructor  competence,  and  how 
well  the  course  had  prepared  the  student  for  the  job. 

Significant  effects  are  present  for  items  relating  to  instructors,  students/teams,  and  interaction/ 
participation. 
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Question 

•  Instructors  1  2  3  4  5 

1.  Instructors  were  adequately  prepared  for  class 

2.  Instructors  presented  lessons  clearly 

3.  Instructors  encouraged  class  participation 

4.  Instructors  answered  student  questions  adequately 

5.  Instructors  made  the  best  use  of  time  available 

6.  Instructors  projected  a  positive  attitude  about  the  subj'ect 

7.  Instructors  could  be  seen  clearly 

8.  Instructors  could  be  heard  adequately 

9.  Instructors  maintained  adequate  control  of  the  class 

•  Students/Teams 

10.  Students  in  other  teams  could  be  seen  clearly 

11.  Students  in  my  own  team  could  be  seen  clearly 

12.  Students  in  other  teams  could  be  heard  adequately 

13.  Students  in  my  own  team  could  be  heard  adequately 

•  Learning  Activities 

14.  Case  studies  provided  a  useful  learning  experience 

15.  Exercises  provided  a  useful  learning  experience 

16.  Simulations  provided  a  useful  learning  experience 

17.  Written  materials  were  clearly  written 


Figure  6.  NAVLEAD  Student  Questionnaire  (Q.2)  responses. 


I  Traditional  I  tej  VTT  Local  \/A  VTT  Remote 
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SCALE: 

1  =  Unsatisfactory 
5  =  Outstanding 
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Question 


•  Training  Aids 

18.  Training  aids  were  vaiuable  in  supporting  instruction 


19.  Training  aids  were  used  effectively 


20.  Details  of  training  aids  could  be  clearly  seen 

•  Interaction/Participation 

21.  Interaction  between  instructors  and  students  was  sufficient 
to  support  learning  objectives 

22.  Interaction  aniong  the  members  of  my  team  was  sufficient 
to  support  learning  objectives 

23.  Interaction  among  the  different  teams  was  sufficient  to  support 
learning  objectives 


24.  Class  participation  was  sufficient  to  support  learning  objectives 

•  Overall 

25.  Compare  the  instructor's)  to  Navy  instructors  who  have  taught 
you  in  the  past 


26.  Compare  the  course  to  other  Navy  courses  you  have  taken 
in  the  past 

27.  Rank  how  well  the  course  provided  you  with  skills  that  can  be 
applied  on  the  job 


Traditional  nviT  Local  Y7P<  VTT  Remote 


1  2  3  4  5 


yyyyyyyyyyyA  4j 

1 . 

I  SCALE: 

1  s  Unsatisfactory 
5  =  Outstanding 


Figure  6.  (Continued). 


Instructors.  Significant  main  effects  were  found  for  items  1,  2,  6,  7,  and  8.  HSD  multiple 
comparison  tests  in  these  cases  indicate  that  differences  between  traditional  and  VTT  local  (i.e.,  T- 
L)  scores  were  not  significant  but  that  differences  between  traditional  and  VTT  remote  (T-R)  were 
significant.  VTT  local  and  VTT  remote  (L-R)  were  also  significantly  different  in  all  but  one  case 
(question  2).  In  these  instances  the  ratings  given  by  students  in  VTT  remote  classrooms  regarding 
instructor  preparation,  projection  of  a  positive  attitude,  visibility,  and  audibility  were  significantly 
lower  than  by  students  in  traditional  or  VTT  local  classrooms. 

Students/teams.  Significant  main  effects  were  found  for  all  items  (10  through  13).  HSD 
comparisons  indicate  significant  differences  in  all  three  group  comparisons,  with  the  largest 
differences  being  found  for  items  10  and  12.  Students  in  VTT  classrooms  (local  or  remote)  could 
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Table  3 


Statistical  Comparisons  for  NAVLEAD  Student  Questionnaire  (Q.2) 


ANOVA  Mean  Diffeences  and 

T-L-R  Tukey  HSD  Comparisons 


Question 

DF 

F 

T-L 

T-R 

L-R 

INSTRUCTORS 

1.  Instructors  were  adequately  prepared  fw  class 

2,102 

5.42** 

0.0 

0.4* 

0.4* 

2.  Instructors  presented  lessons  clearly 

2,102 

4.26* 

0.1 

0.4* 

0.2 

3.  Instructors  encouraged  class  participation 

2,102 

2.10 

0.1 

0.2 

0.1 

4.  Instructors  answered  student  questions  adequately 

2,101 

0.01 

0.0 

0.0 

0.0 

5.  Instructors  made  the  best  use  of  time  available 

2,102 

2.72 

0.2 

0.4 

0.2 

6.  Instructors  projected  a  positive  attitude  about  the  subject 

2,102 

5.44** 

0.1 

0.3* 

0.3* 

7.  Instructors  could  be  seen  clearly 

2,102 

22.01** 

0.1 

0.7* 

0.7* 

8.  Instructors  could  be  heard  adequately 

2,102 

11.63** 

0.1 

0.6* 

0.5* 

9.  Instructors  maintained  adequate  control  of  the  class 

2,102 

2.00 

-0.2 

0.1 

0.3 

STUDENTSATEAMS 

10.  Students  in  other  teams  could  be  seen  clearly 

2,101 

13.86** 

0.6* 

1.1* 

0.5 

11 .  Students  in  my  own  team  could  be  seen  clearly 

2,100 

8.11** 

0.3 

0.6* 

0.3 

12.  Students  in  other  teams  could  be  heard  adequately 

2,101 

17.15** 

0.4* 

1.2* 

0.8* 

13.  Students  in  my  own  team  could  be  beard  adequately 

2,101 

6.55** 

0.3* 

0.6* 

0.2 

LEARNING  ACTIVITIES 

14.  Case  studies  provided  a  useful  learning  experience 

2,10l‘ 

0.06 

0.0 

0.0 

0.0 

15.  Exercises  provided  a  useful  learning  experience 

2,101 

0.56 

0.0 

0.2 

0.1 

16.  Simulations  provided  a  useful  learning  experience 

2,101 

0.84 

0.2 

0.2 

0.0 

17.  Written  materials  were  clearly  written 

2,101 

1.74 

0.1 

0.4 

0.3 

TRAINING  AIDS 

18.  Training  aids  were  valuable  in  supporting  instruction 

2,102 

2.59 

0.1 

0.4 

0.3 

19.  Training  aids  wwe  used  effectively 

2,102 

2.72 

0.0 

0.5 

0.4 

20.  Details  of  training  aids  could  be  clearly  seen 

2,102 

1.66 

0.1 

0.3 

0.2 

INTERACTION/PARTICIPATION 

21.  Interaction  between  instructors  and  students 
was  sufficient  to  support  learning  objectives 

2,102 

5.64** 

0.1 

0.5* 

0.3 

22.  Interaction  among  the  members  of  my  team 
was  sufficient  to  support  learning  objectives 

2,102 

4.07* 

0.3 

0.4 

0.1 

23.  Interaction  among  the  different  teams 

was  sufficient  to  support  learning  objectives 

2,102 

12.08** 

0.4* 

0.9* 

0.5* 

24.  Class  participation  was  sufficient  to 
suppmt  learning  objectives 

2,102 

9.61** 

0.2 

0.6* 

0.4* 

OVERALL 

25.  Clompare  the  instructor(s)  to  Navy  instructors 
who  have  taught  you  in  the  past 

2,102 

1.23 

0.1 

0.2 

0.2 

26.  Compare  the  course  to  other  Navy  courses  you 
have  taken  in  the  past 

2,102 

0.80 

0.0 

0.2 

0.2 

27.  Rank  how  well  the  course  provided  you  with  skills 
that  can  be  applied  on  the  job 

2,101 

0.79 

-0.1 

-0.2 

-0.1 

Note.  Scale  is  1-5  for  mean  differences:  T-L  =  Traditional-Local;  T-R  =  Traditional-Remote;  L-R  =  Local-Remote 
♦p  <  .05  ’•‘♦p  <  .01  (Tukey  HSD  comparisons  were  judged  signiOcant  at  p  <  .05). 
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not  see  or  hear  members  of  other  teams  as  well  as  students  in  traditional  classrooms.  Interestingly, 
a  somewhat  similar,  but  less  pronounced,  pattern  of  responses  was  found  for  questions  1 1  and  13 
(relating  to  the  student’s  own  team),  indicating  that  students  in  VTT  classrooms  could  not  hear 
members  of  their  own  team  as  well  as  students  in  traditional  classrooms.  The  remote  students  also 
reported  a  relative  difficulty  in  seeing  their  own  team  members.  The  possible  source  of  these 
findings  is  the  different  furniture  and  seating  arrangement  in  VTT  classrooms  (see  Figure  1),  in 
which  VTT  students  did  not  face  one  another. 

Interaction/partidpation.  Significant  main  effects  were  found  for  items  21,  22,  23,  and  24. 
HSD  multiple  comparison  tests  in  most  of  these  cases  indicated  that  differences  between  traditional 
and  VTT  local  scores  were  smaller  than  differences  between  traditional  and  VTT  remote  or 
between  VTT  local  and  VTT  remote  classrooms.  The  pattern  of  these  differences  indicates  that 
ratings  given  by  students  in  VTT  remote  classrooms  regarding  student  interaction  and  participation 
were  lower  than  by  students  in  traditional  or  VTT  local  classrooms. 

Open-Ended  Questions 

Questions  28, 29,  and  30  on  Q.2  asked,  respectively,  what  students  liked  most  about  the  course, 
liked  least,  and  for  any  suggestions  to  improve  the  course.  A  total  of  105  students  completed  the 
questionnaire  (47  traditional,  36  VTT  local,  22  VTT  remote).  Responses  to  all  three  questions  were 
exhaustively  listed  and  clustered  into  6  broad  categories,  and  non  responses  were  placed  in  a  “no 
comment”  category. 

Figure  7  summarizes  the  results  for  question  28.  A  total  of  113  comments  was  made  by  96 
students  for  an  overall  response  rate  of  91%  and  with  all  groups  responding  at  this  level.  The  level 
of  responses  by  category  differ  among  groups  but  do  not  appear  to  be  systematic.  The  four  most 
common  types  of  comments  related  to  interaction,  instructional  strategies,  subject  matter,  and 
instructors,  all  of  which  reflect  the  course  rather  than  the  delivery  medium;  one  way  to  encapsulate 
this  pattern  is  to  observe  that  what  students  liked  most  about  the  course  was  some  aspect  of  the 
course  itself. 

Figure  8  summarizes  the  results  for  question  29.  A  total  of  105  comments  was  made  by  70 
students  for  a  response  rate  of  67%.  The  response  rate  was  lowest  for  traditional  (57%),  higher  for 
VTT  local  (69%),  and  highest  for  VTT  remote  (82%),  indicating  an  increasing  tendency  across 
these  groups  for  students  to  identify  things  they  did  not  like  about  the  course.  The  levels  of 
responses  by  category  differ  among  groups  but  do  not  appear  to  be  systematic.  Instructional 
strategies  were  cited  by  nearly  a  quarter  of  students  in  the  traditional  group  and  by  much  smaller 
percentages  in  VTT  groups;  the  cause  of  this  unexpected  result  is  unclear.  Fairly  high  percentages 
of  students  in  all  groups  made  comments  relating  to  course  subject  matter.  VTT  and  equipment- 
related  comments  were  made  by  nearly  half  of  students  in  the  VTT  remote  group,  a  far  higher 
percentage  than  in  other  groups.  The  thing  VTT  remote  students  were  most  likely  to  dislike  related 
to  the  delivery  medium,  a  factor  that  mattered  far  less  to  students  in  the  other  two  groups. 

Figure  9  summarizes  the  results  for  question  30.  A  total  of  1 15  comments  was  made  by  55 
students  for  a  response  rate  of  52%.  The  response  rate  was  lowest  for  traditional  (40%),  higher  for 
VTT  local  (53%),  and  highest  for  VTT  remote  (77%),  repeating  the  pattern  found  with  Figure  8 
and  indicating  an  increasing  tendency  across  these  groups  for  students  to  make  suggestions  for 
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Interaction  Instructional  VTT&  Subject  Instructors  Facilities  No 

strategies  equipment  matter/  Comment 

materials 


Comments  by  Category 


Figure  8.  Student  responses  to  item  29  on  NAVLEAD  Student  Questionnaire  (Q.2) 
(“What  did  you  like  the  least  about  this  course?”). 
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improving  the  course.  The  level  of  responses  by  category  differ  among  groups  systematically  only 
for  the  VTT  and  equipment  category,  where  VTT  remote  students  were  the  most  likely  to  offer 
suggestions. 
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Figure  9.  Student  responses  to  item  30  on  NAVLEAD  Student  Questionnaire  (Q.2) 
(“Discuss  any  suggestions  you  have  for  improving  the  course.”). 


NAVLEAD  Quiz  (Q.3) 

The  NAVLEAD  Quiz  (Appendix  D)  consists  of  25  four-alternative  multiple-choice  items 
covenng  course  content.  The  quiz  was  administered  at  the  beginning  of  the  course  as  a  pretest  and 
at  the  end  of  the  cour^  as  a  posttest.  Pretest,  posttest,  and  difference  (posttest-pretest)  percent 
correct  scores  for  Traditional,  VTT  local,  and  VTT  remote  groups  are  shown  in  Table  4.  Pretest, 
posttest,  and  difference  scores  are  very  close,  regardless  of  group,  and  differences  among  groups 
are  not  statistically  significant . 


Table  4 

Student  Scores  on  NAVLEAD  Quiz  (Q.3) 


Class  Type 


No.  Students 


Pretest 


IVaditional 
VTT  Local 
VTT  Remote 


Posttest 


47 

36 

22 


51.25  (9.72) 
52.33  (10.10) 
52.18  (11.55) 


Difference 


62.67  (9.91) 
63.56  (12.63) 
64.18  (12.44) 


Standard  deviations  for  percent  correct  scores  are  in  parentheses. 


11.42 

11.22 

11.00 
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Facilitator/Observer  Questionnaire  (Q.4) 


Appendix  E).  Hie  raungs  were  f*  »S.Sng”  Fourteen  obLrvers  conrpleted  Q.4 

indicating  “Unsatisfactory”  and  5  indicating  Outstanaing.  limiting 

Lng  dre  tonr  oourae  convenings.  Only  one  ob^er  - 

f,'?? between-snbjects  variable)  for  qoestio^  1, 2 

?1T7  kTd  W  ta^qSons!.w"groupoornparisonsweren.adebetwe^^ 

LWlcirnbininglocalandrernoref-Nogroup^mpanso^W 

it  applied  only  J“  ^  were  stmetored  in  two  parts;  part  “a”  relating 

f’  H  V  ti  vSmote.  Thus,  VTT  course  observers,  regardless  of  locauon,  were 

rtle  to  Se  dimensions  of  both  local  and  ternote  '(“^^^vtdm^dam  ena^n;^^ 

re;^:sras!rerto  "r^p^prolms^  on  different  qu^ity  dimensions  during 

the  week. 

™  rSg  aleTL.  observers  gave  positive  ratings  to  the  dimension  bemg  measured. 


A  few  trends  are  apparent  throughout  the  items  in  Figure  10: 


•  Highest  ratings  are  for  traditional 

.  Higher  ratings  are  usually  given  for  VTT  local  than  VTT  remote 
.  Ratings  tend  to  improve  during  the  week  for  VTT  classes  but  do  not  usually  reach  parity 
with  traditional  classes 

.  Ratings  for  traditional  classes  show  little  or  no  improvement  during  the  week 

These  figures  can  be  further  interpreted  by  considering  the  results  of  statistical  analyses  Aat 
These  tigur^  c^ne  iu  v  conducted  to  determine  the  statistical 

■gl'^nfrZg  diteret-s  for  dre  between-  and 
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Rating  P-  Rating  P  Rating  FI  Rating 


&  Question  1:  Effectiveness  of  instructor’s  presentation  e. 

S| - T - T - - T -  5i 

4 - V.. . -V-.,  ^ . -.1 - Y -  4} 


Question  5:  Degree  of  instructor  control 


Mon  Tue  Wed  Thur  Fr 
Day  of  the  Week 


Question  2:  Effectiveness  of  case  studio 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Question  3:  Effectiveness  of  exercises 

. - T - 


g.  Question  7:  Amount  of  instructor-student  interaction 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Question  4:  Effectiveness  of  simulations 

- T - T - T— ^T - T- 


Tue  Wed  Thur  Fri 
Day  of  the  Week 


Legend 


T  =  Traditional 
V  =  VTT 
LsVTT  Local 
R  =  VTT  Remote 


SCALE: 


Tue  Wed  Thur 
Day  of  the  Week 


1  s  Unsatisfactory 
5  =  Outstanding 
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Rating  |  Rating  Rating  Rating 


h.  Question  8:  Success  In  meeting  iearning  objectives  1 1.  Question  12:  Amount  of  student  interaction 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Questbn  9:  Amount  of  student  participation 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


n.  Question  14:  Amount  of  instructor-student  interaction 


Mon  Tue  Wed  Thur  Fri 
Day  of  the  Week 


Legend 


T  =  Traditional 
V  =  VTT 
L  =  VTT  Local 
R  =  VTT  Remote 


SCALE: 


1  =  Unsatisfactory 
5  =  Outstanding 
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VTT  local).  The  remaining  columns  give  the  degrees  of  freedom  and  F  ratio  ANOVA  results  for 
the  treatment  group  variable,  day  of  week,  and  their  interaction. 

Table  5 


Analysis  Results  for  NAVLEAD  Fadlitator/Observer  Questionnaire  (Q.4). 


Question 

Between 

Groups 

Comparison 

Between  Groups 

DF  F 

ANOVA 

Day  of  Week 

DF  F 

Interaction 

DF  F 

1  Effectiveness  of  instructor’s  presentations 

Tvs  VTT 

1,12 

5.64* 

4,48 

0.00 

4,48 

0.00 

2.  Effectiveness  of  case  studies 

Tvs  VTT 

1,10 

5.49* 

4,40 

3.61* 

4,40 

0.00 

3.  Effectiveness  of  exercies 

Tvs  VTT 

1,6 

2.20 

4,24 

3.53* 

4,24 

0.00 

4.  Effectiveness  of  simulations 

Tvs  VTT 

1,4 

1.33 

4.4 

3.14 

4,4 

1.57 

5.  Degree  of  instructor  control 

T  VS  L  VS  R 

2,19 

3.89* 

4,76 

4.31** 

8,76 

1.27 

6.  Difficulty  of  conducting  instruction 

Tvs  VTT 

1,10 

4.55 

4,40 

4.36** 

4,40 

1.99 

7.  Amount  of  instructor-student  interaction 

Tvs  VTT 

1,11 

9.26** 

4,44 

2.11 

4,44 

1.32 

8.  Success  in  meeting  learning  objectives 

Tvs  VTT 

1,12 

4.73* 

4,48 

3.02* 

4,48 

1.68 

9.  Amount  of  student  participaticm 

T  vs  L  vs  R 

2,19 

11.29** 

4,76 

9.53** 

8,76 

2.66* 

10.  Amourn  of  interaction  among  teams 

Tvs  VTT 

1,7 

8.28* 

4,28 

2.64 

4,28 

2.02 

11.  Degree  of  team  cohesiveness 

T  vs  L  vs  R 

2,13 

7.25** 

4,52 

8.66** 

8,52 

0.00 

12.  Amount  of  student-student  interaction 

T  vs  L  vs  R 

2,17 

4.26* 

4,68 

15.58** 

8,68 

1.95 

13.  Student  interaction  between  classes 

N/A  -  Days 

N/A 

N/A 

4,32 

6.69** 

N/A 

N/A 

14.  Quality  of  student  presentations 

T  vs  L  vs  R 

2,12 

10.45** 

4,48 

6.55** 

8,48 

0.00 

*p<.05 

**p  <  .01 


Traditional  vs  VTT 

Significant  effects  were  found  for  all  items  except  3, 4,  and  6  (few  observers  responded  to  items 
3  and  4).  With  these  exceptions,  observers  gave  higher  ratings  for  traditional  than  VTT,  regardless 
of  dimension.  The  effects  were  strongest  for  dimensions  directly  bearing  upon  student  participation 
(e.g.,  7, 9, 10, 1 1, 14),  and  weaker  on  dimensions  more  abstractly  defined  or  relying  primarily  upon 
the  instructor. 

Day  of  Week 

Significant  day  of  week  effects  were  found  for  10  of  the  14  items.  The  effect  for  item  1 
(effectiveness  of  the  instructor’s  presentation)  was  not  significant;  ratings  on  this  dimension  did 
not  change  much  during  the  week,  although  higher  ratings  were  always  given  in  traditional  than 
VTT  classes.  The  effect  for  item  4  (effectiveness  of  simulations)  was  not  significant  (few  observers 
rated  this  item).  The  effect  on  item  7  (amount  of  instructor-student  interaction)  was  not  statistically 
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significant,  although  Figure  lOg  shows  some  growth  for  VTT  during  the  week.  Again,  effects  were 
strongest  on  items  directly  bearing  upon  student  participation  (e.g.,  9, 1 1, 12, 13, 14)  and  on  item 
5  (degree  of  instructor  control).  These  results  suggest  an  improvement  in  training  participants’ 
ability  to  interact  with  one  another  oh  different  levels  during  a  week  of  training. 


Interactions 

A  significant  interaction  indicates  that  the  difference  between  the  treatment  groups  changed  in 
some  manner  over  the  days  of  the  week.  A  significant  interaction  was  found  only  for  item  9.  Figure 
lOi  indicates  ihai  this  rating  on  the  amount  of  student  participation  was  more  likely  to  inaprove  for 
the  VTT  remote  group  than  for  the  other  two  groups.  Items  4  and  12  also  suggest  some  differential 
improvement  for  VTT  groups  over  the  week,  but  no  significant  interaction  was  obtained  for  these 

items. 

Performance  Activities  (Q.5) 

The  performance  activities  upon  which  Q.5  is  based  were  not  conducted  in  any  of  the  four 
courses  observed,  although  they  were  called  for  in  the  instructor  s  guide.  Consequently,  it  was  not 
possible  to  obtain  this  form  of  objective  performance  data  for  the  course. 

Class  Participation  Tally  (Q.6) 

Class  participation  was  assessed  by  having  an  observer  record  on  Q.6  (Appendix  G)  the 
number  of  student-initiated  questions  and  comments  given  by  each  student  team.  These  were 
recorded  daily  during  two  one-hour  intervals  (0900-1000  and  1300-1400).  Six  observers 
completed  Q.6,  with  an  observer  present  in  each  of  two  traditional,  VTT  local,  and  VTT  remote 
classrooms.  The  data  enable  three-group  between-subjects  comparisons,  and  since  the  Q.6  tally 
was  completed  on  a  daily  basis,  it  also  provides  a  set  of  daily  within-subjects  repeated  measures  to 
compare  progress  during  the  week. 

The  number  of  interactions  for  the  two  daily  hours  of  recording  were  averaged  for  each  team 
and  analyzed  to  determine  how  level  of  interaction  varied  as  a  function  of  group  and  day  of  week. 
Thus,  the  numbers  discussed  below  reflect  the  average  number  of  interactions  from  a  student  team 
in  a  one-hour  period.  Results  are  shown  in  graphic  form  in  Figures  11  and  12.  Several  trends  are 
apparent  in  the  figures,  the  most  noticeable  being  that  the  overall  level  of  interaction  for  traditional 
is  about  twice  that  for  VTT.  However  this  trend  is  mitigated  by  some  variability  shown  between 
the  two  traditional  classes.  Over  days,  the  level  of  interaction  for  VTT  appears  to  rise  slightly  more 
during  the  week  than  for  traditional. 

An  analysis  of  variance  indicated  that  the  between-subjects  treatment  group  variable 
(traditional,  VTT  local,  or  VTT  remote)  was  significant  (F(2,16)  =  7.33,  p  <  .01).  This  reflects  a 
greater  level  of  interaction  for  the  traditional  group  than  in  the  VTT  groups.  An  overall  increase  in 
the  level  of  interaction  over  the  week  was  indicated  by  the  day  of  week  main  effect,  which  was 
approximately  at  the  level  of  significance  (F(4,64)  =  2.46,  p  =  .054).  The  statistical  interaction 
between  treatment  group  and  day  of  week  was  significant  (F(8,64)  =  2.49,  p  <  .05).  This  indicates 
that  the  level  of  the  treatment  groups  differed  over  the  week,  i.e.  the  inconsistent  level  on  Tuesday 
for  the  traditional  condition  and  the  increasing  level  for  the  VTT  conditions. 
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Figure  11.  Level  of  student  interaction  as  indicated  by  Class  Participation  Tally  (Q.6)  by 
group  and  day  of  week. 


Figure  12  shows  the  level  of  interaction  (per  hour  per  team)  averaged  over  the  week.  The  bars 
with  dotted  lines  show  the  average  of  the  two  classes  per  treatment  condition,  and  the  solid  bars 
within  these  show  the  levels  for  the  two  individual  classes  in  a  treatment  condition.  The  average  of 
the  two  traditional  classes  was  almost  twice  that  for  the  VTT  classes,  with  the  VTT  local  class 
showing  slightly  more  interaction  than  the  VTT  remote  class.  However,  there  was  substantial 
variability  between  the  two  traditional  classes,  with  one  showing  twice  the  level  of  the  other.  With 
a  data  set  this  small,  it  is  not  possible  to  attribute  this  variability  to  differences  among  the  observers, 
classes,  or  instructors. 

Readers  should  be  aware  that  this  analysis  may  be  handicapped  by  the  fact  that  the  number  of 
students  and  teams  contributing  to  the  interaction  tallies  were  not  equivalent  among  the  traditional 
and  VTT  treatment  groups.  The  analysis  above  reported  interactions  from  teams  and  the  average 
number  of  teams  varied  by  treatment  group:  4  (traditional)  and  5.5  (VTT  combined).  The  average 
number  of  students  also  varied  by  treatment  group:  23.5  (traditional)  and  29.0  (VTT  combined). 
One  might  expect  amount  of  interaction  to  decrease  (a)  with  more  students  or  teams  since  more 
people  must  share  the  time  available  for  interaction,  or  (b)  as  a  consequence  of  some  inhibition  due 
to  use  of  the  VTT  technology.  However,  the  differences  in  level  of  student  interaction  indicated  in 
Figure  1 1  and  12  are  greater  than  might  be  expected  from  the  number  of  students  alone.  Practically, 
it  may  not  matter  whether  the  reduction  in  student  interaction  is  caused  by  VTT  or  class  size  since 
VTT  classes  generally  have  more  students  than  traditional  classes. 
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Figure  12.  Student  interaction  on  Class  Participation  Tally  (Q.6)  for  each  individual  class 
within  a  treatment  group. 


Discussion  and  Conclusions 

The  primary  objective  of  the  research  was  to  test  the  feasibility  of  using  VTT  to  deliver  Navy 
leadership  training.  Was  feasibility  demonstrated?  In  purely  technical  terms,  the  answer  is  yes.  The 
classes  were  conducted  successfully,  students  received  training  and  graduated,  and  there  was  no 
significant  outcry  about  the  way  their  training  was  being  received. 

Student  responses  on  questionnaires  tended  to  favor  traditional  instruction  slightly,  but 
differences  were  small  (Q.l  and  Q.2).  Students  did,  however,  express  the  opinion  that  VTT 
reduced  their  opportunities  to  interact  with  both  the  instructor  and  other  students.  However,  no 
significant  differences  were  found  with  the  NAVLEAD  quiz  (Q.3)  on  knowledge  of  course 
content.  Subject  matter  expert  ratings  of  various  dimensions  of  the  training  (on  Q.4)  pr^ent  a 
slightly  different  picture.  On  virtually  every  dimension,  significantly  higher  ratings  were  given  to 
traditional  instruction  than  VTT  and,  though  improvement  occurred  throughout  the  week,  VTT 
classes  did  not  reach  parity  with  traditional  classes.  The  class  participation  tally  (Q.6)  indicated 
that  the  level  of  interaction  in  VTT  classes  was  lower  than  in  traditional  classes.  Thus,  interactivity 
was  reduced  in  VTT  classes  as  shown  by  the  expressed  perceptions  of  students  and  subject  matter 
experts  and  by  an  objective  count  of  interactions,  but  course  knowledge  was  not  affected. 

In  several  respects,  the  use  of  VTT  forced  trainers  to  make  compromises  in  terms  of  factors  Aat 
the  Navy  leadership  community  has  valued:  instructor-student  and  student-student  interaction 
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were  reduced,  the  intensity  of  the  learning  environment  and  ability  of  instructors  to  perceive 
remote  students’  nonverbal  cues  were  lessened  by  the  limited  view  offered  through  the  VTT 
system,  and  some  experiential  learning  experiences  were  made  more  difficult  to  conduct  with 
VTT.  The  data  from  subject  matter  expert  ratings  and  that  pertaining  to  participation  suggest  that 
VTT  led  to  some  reduction  in  the  interactivity  of  the  NAVLEAD  learning  environment  as 
compared  to  that  in  traditional  classrooms  in  Little  Creek  or  Coronado. 

Do  these  findings  and  judgments  mean  that  VTT  should  not  be  used  for  NAVLEAD 
instruction?  On  the  one  hand,  the  data  reported  here  suggest  some  reduction  in  interactivity  and 
perceived  quality.  On  the  other  hand,  it  is  not  clear  that  the  course  has  been  compromised,  and 
several  other  considerations  may  play  in  the  decision.  Ultimately,  weighing  these  factors  and  the 
decision  to  teach  NAVLEAD  is  left  to  those  who  have  direct  oversight  of  the  course,  who  pay  for 
it,  and  who  use  its  graduates. 

Some  of  the  other  factors  that  may  need  to  be  weighed  in  a  decision  to  teach  NAVLEAD  on 
VTT  are  as  follows.  First,  the  differences  between  traditional  and  VTT  NAVLEAD  classes  mainly 
reflect  subjective  perceptions  of  reduced  quality.  Additionally,  these  differences  are  still  generally 
above  the  mid  point  of  the  rating  scales.  Second,  the  data  reported  here  are  preliminary  and  reflect 
a  first  attempt  to  deliver  NAVLEAD  instruction  via  VTT.  Two  additional  NAVLEAD  courses 
have  since  been  delivered  with  VTT  (Leading  Petty  Officer  and  Chief  Petty  Officer  courses). 
Research  on  those  courses  included  an  evaluation  of  student  performance  during  a  simulated 
classroom  activity  that  will  be  reported  subsequently  (Wetzel,  Simpson,  &  Seymour,  1995).  Third, 
the  delivery  of  the  course  on  VTT  would  likely  undergo  some  improvement  as  instructors  become 
more  experienced  with  VTT  and  develop  new  techniques  to  foster  interaction.  In  addition  to  the 
potential  of  developing  new  instructional  strategies  and  instructor  behaviors  to  encourage  greater 
student  participation,  new  technologies  that  show  better  views  of  individuals  between  sites  could 
also  be  employed.  Fourth,  practical  reasons  to  consider  VTT  for  these  courses  grow  out  of  the  need 
to  reduce  costs  and  address  shortages  of  instructors  due  to  a  drawdown.  Use  of  the  CNET 
Electronic  Schoolhouse  Network  (CESN)  VTT  system  can  favorably  impact  the  cost  of  training 
by  reducing  the  costs  associated  with  travel,  per  diem,  and  instructors.  Use  of  the  system  is 
particularly  cost-beneficial  for  courses  with  a  high  throughput  that  are  short  in  duration  (a  week  or 
less). 


Recommendations 

1.  In  making  its  decision  to  use  VTT  for  NAVLEAD,  the  Chief  of  Naval  Education  and 
Training  should  consider  both  this  evaluation  and  that  subsequently  conducted  for  LPO  and  CPO 
courses. 

2.  If  the  decision  is  made  to  conduct  NAVLEAD  instruction  with  VTT,  the  Chief  of  Naval 
Education  and  Training  should  test  and  refine  ways  to  foster  higher  levels  of  instructor-student  and 
student-student  interaction. 
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Appendix  A 

Data  Collection  Instructions 
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Data  Collection  Instructions 


Overview 

The  site  coordinator  is  responsible  for  administering  three  data  collection  instru¬ 
ments: 

1.  VTT  Student  Questionnaire  (Q.l) 

2.  NAVLEAD  Student  Questionnaire  (Q.2) 

3.  NAVLEAD  Quiz  (Q.3) 

The  site  coordinator  must  also  assure  that  facilitators  and  SME  observers  complete 
three  data  collection  instruments: 

1.  Facilitator/Observer  Questionnaire  (Q.4) 

2.  Performance  Activities  (Q.5) 

3.  Class  Participation  Tally  (Q.6) 

The  purpose,  content,  completion  requirements,  and  time  required  for  each  of  these 
instruments  are  described  below. 

Facilitators  and  observers  should  review  each  instrument  to  become  familiar  with 
instructions  and  content.  Facilitators  and  observers  should  keep  a  personal  log  of 
"significant"  events  observed  during  training.  The  log  should  record  the  type  of  event, 
time,  and  comments  for  discussion  during  the  course  critique  following  the  course. 


Data  Collection  Instruments 

VTT  Student  Questionnaire  (Q.l) 

Purpose:  Assess  student  perceptions  of  VTT  quality. 

Content:  Ratings  of  video,  audio,  VTT  procedures,  local  vs.  remote  team  partici¬ 
pation;  and  multiple-choice  items  regarding  student  preferences. 

Completed  by:  Students  (VTT  only) 

When  completed:  At  conclusion  of  training 
Time  required:  5  min. 

NAVLEAD  Student  Questionnaire  (Q.2) 

Purpose:  Assess  student  perceptions  of  training  quality. 

Content:  Ratings  of  facilitator,  personnel  visibility  &  audibility,  written  materials, 
learning  activities,  training  aids,  interaction/participation,  overall  evaluation;  and 
open-ended  questions  about  student  likes,  dislikes,  and  suggestions. 

Completed  by:  Students  (both  traditional  and  VTT) 

When  completed:  At  conclusion  of  training 
Time  required:  10  min. 
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NAVLEAD  Quiz  (Q.3) 

Purpose:  Assess  student  knowledge  of  course  content  pre/post  course. 

Content:  Consists  of  28-item  four-choice  multiple  choice  test  covering  course 
content. 

Completed  by:  Students 

When  completed:  Before  course  (pre),  after  course  (post). 

Time  required:  25  min,,  25  min, 

Facilitator/Observer  Questionnaire  (Q.4) 

Purpose:  Assess  facilitator/observer  perceptions  of  training  quality. 

Content:  Ratings  on  various  dimensions  of  the  course  from  facilitator/SME  per¬ 
spective  (e.g.,  effectiveness  of  presentations  and  exercises,  success  in  meeting 
learning  objectives,  difficulty  of  conducting  instruction,  student  interaction,  student 
participation,  degree  of  control,  presentation  quality,  cohesiveness;  space  is  pro¬ 
vided  for  comments. 

Completed  by:  ALL  facilitators  and  SME  observers  (both  traditional  and  VTT) 
When  completed:  Daily 
Time  required:  10  min./day 

Performance  Activities  (Q.5) 

Purpose:  Assess  student  performance. 

Content:  Facilitator/SME  ratings  on  several  different  dimensions  of  student  per¬ 
formance  in  completing  classroom  performance  activities  for  lesson  3.5 
(DIVO/CPO  relationships)  and  5.4  (assessing  the  division). 

Completed  by:  ALL  facilitators  and  SME  observers  (both  traditional  and  VTT) 
When  completed:  Immediately  following  performance  activity  (day  3  and  day  5) 
Time  required:  5  min.,  10  min. 

Class  Participation  Tally  (Q.6) 

Purpose:  Assess  class  participation. 

Content:  Frequency  tally  of  student-initiated  questions  and  comments  by  team 
across  time.  Remarks  must  be  directed  at  facilitators  or  students,  relate  to  course 
content,  and  intended  for  class  to  hear. 

Completed  by:  ONE  SME  observer  (both  traditional  and  VTT) 

When  completed:  During  class 

Time  required:  2  hrs/day  (concurrent  with  class) 
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Daily  Data  Collection  Schedule 


- Monday - 

0700-0725  (before  class  starts):  Administer  Q.3  to  all  students.  Allow  students  25 
minutes  to  complete  and  then  collect. 

0900-1000  &  1300-1400:  Complete  Q.6. 

1500-1600  (after  class):  Complete  Q.4  for  day  1. 

- Tuesday - 

0900-1000  &  1300-1400:  Complete  Q.6. 

1500-1600  (after  class):  Complete  Q.4  for  day  2. 

- Wednesday - 

0900-1000  &  1300-1400:  Complete  Q.6. 

Complete  Q.5  for  lesson  3.5  during  or  immediately  after  the  DIVO/CPO  relationships 
lesson. 

1500-1600  (after  class):  Complete  Q.4  for  day  3. 

- Thursday - 

0900-1000  &  1300-1400:  Complete  Q.6. 

1500-1600  (after  class):  Complete  Q.4  for  day  4. 

- - Friday - 

0900-1000  &  1300-1400:  Complete  Q.6. 

Complete  Q.5  for  lesson  5.4  during  or  immediately  after  the  "assessing  the  division"  les¬ 
son. 

1500-1600  (after  class):  Complete  Q.4  for  day  5. 

1500-1600  (after  class):  Administer  Q.3  to  all  students.  Allow  students  25  minutes  to 
complete  and  then  collect. 

1500-1600  (after  class):  Administer  Q.2  to  all  students.  There  is  no  fixed  time  limit  but 
they  should  take  about  10  minutes. 
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VTT  Student  Questionnaire  (Q.l) 
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VTT  STUDENT  QUESTIONNAIRE  (Q.l) 


1 .  Name^ _ 

Last  First  MI  Rank 

2.  Today's  date _ / _ / _  3.  Location  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 


For  each  of  the  following  statements  (1  through  17),  check  the 
appropriate  box  corresponding  to  a  scale  of  1  (unsatisfactory)  through  5 
(outstanding),  with  3  being  average. 


VIDEO 


Unsatisfactory — >Outstanding 


1 .  Image  on  TV  was  large  enough  to  be  seen 

[1] 

[2] 

[3] 

[4] 

[5] 

2 .  Image  on  TV  was  clear  enough  to  be  seen 

[1] 

[2] 

[3] 

[4] 

[5] 

3.  Graphics/slides/transparencies  on  TV  were  readable 

[1] 

[2] 

[3] 

[4] 

[5] 

AUDIO 

4,  Audio  from  other  class  was  loud  enough  to  understand 

[1] 

[2] 

[3] 

[4] 

[5] 

5 .  Audio  from  other  class  was  clear  enough 
to  understand 

[1] 

[2] 

[3] 

[4] 

[5] 

6 .  Instructor' s  voice  could  be  heard  adequately 

[1] 

[2] 

[3] 

[4] 

[5] 

7 .  Student  voices  could  be  heard  adequately 

[1] 

[2] 

[3] 

[4] 

[5] 

8 .  Microphones  were  convenient  to  use 

[1] 

[2] 

[3] 

[4] 

[5] 

VTT  PROCEDURES 

9 .  Students  knew  how  to  use  microphones 

[1] 

[2] 

[3] 

[4] 

[5] 

10.  Students  knew  how  to  stay  on  camera 

[1] 

[2] 

[3] 

[4] 

[5] 

11.  Students  knew  how  to  attract  instructor's 
attention 

[1] 

[2] 

[3] 

[4] 

[5] 

12.  Instructors  handled  q[uestions /comments  effectively 

[1] 

[2] 

[3] 

[4] 

[5] 

13.  Instructors  coordinated  activities  among  classes 
effectively 

[1] 

[2] 

[3] 

[4] 

[5] 

14.  VTT  technical  problems  were  resolved 
in  a  timely  manner 

[1] 

[2] 

[3] 

[4] 

[5] 

15.  Instructors  were  prepared  to  teach  on  the 

VTT  network 

[1] 

[2] 

[3] 

[4] 

[5] 
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Unsatisfactory — >Out standing 

INTERACTION/PARTICIPATION 


16.  There  was  good  cohesiveness  between  local 


and  remote  sites 

[1] 

[2] 

[3] 

[4] 

[5] 

Local  and  remote  sites  participated  equally  in  the 
class 

[1] 

[2] 

[3] 

[4] 

[5] 

STUDENT  PREFERENCES 


18.  How  did  the  VTT  method  of  instruction  affect  your  opportunities  to 
interact  with  the  instructor? 

[ ]  more  opportunities 

[]  no  effect  on  opportunities 

[]  fewer  opportunities 

19.  How  did  the  VTT  method  of  instruction  affect  your  opportunities  to 
interact  with  other  students? 

[]  more  opportunities 

[]  no  effect  on  opportunities 

[]  fewer  opportunities 

20.  Which  method  of  instruction  would  you  have  preferred  for  this  course? 

[]  VTT 

[]  Traditional  (live)  instruction 

[]  No  preference  between  VTT  and  traditional  instruction 

21.  Which  of  the  following  would  you  prefer?  (check  one) 

[]  Enrolling  in  a  VTT  course  near  your  home  port? 

[]  Enrolling  in  a  traditional  (live)  course  farther  (TAD)  from 
your  home  port? 

22.  Which  of  the  following  would  you  prefer?  (check  one) 

[]  Enrolling  in  a  VTT  course  at  a  time  convenient  to  your  time 
schedule? 

[]  Enrolling  in  a  traditional  (live)  course  at  a  time 
inconvenient  to  your  time  schedule? 

23.  If  you  had  a  choice,  would  you  take  another  VTT  course? 

[]  Yes 
[]  No 

Please  explain: _ 
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Appendix  C 

NAVLEAD  Student  Questionnaire  (Q.2) 


c-o 


NAVLEAD  STUDENT  QUESTIONNAIRE  (Q.2) 


1 .  Name _ 

Last  First  MI  Rank 

2.  Today's  date _ / _ / _  3.  Location  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


For  each  of  the  following  statements  (1  through  21),  check  the 
appropriate  box  corresponding  to  a  scale  of  1  (unsatisfactory)  through  5 
(outstanding),  with  3  being  average. 


Unsatisfactory — >Out standing 

INSTRUCTORS 


1.  Instructors 

were  adequately  prepared  for  class 

[1] 

[2] 

[3] 

[4] 

[5] 

2  .  Instructors 

presented  lessons  clearly 

[1] 

[2] 

[3] 

[4] 

[5] 

3 .  Instructors 

encouraged  class  participation 

[1] 

[2] 

[3] 

[4] 

[5] 

4 .  Instructors 

answered  student  questions  adequately 

[1] 

[2] 

[3] 

[4] 

[5] 

5.  Instructors 

made  the  best  use  of  time  available 

[1] 

[2] 

[3] 

[4] 

[5] 

6.  Instructors  projected  a  positive  attitude  about 
the  subject 

[1] 

[2] 

[3] 

[4] 

[5] 

7.  Instructors 

could  be  seen  clearly 

[1] 

[2] 

[3] 

[4] 

[5] 

8 .  Instructors 

could  be  heard  adequately 

[1] 

[2] 

[3] 

[4] 

[5] 

9.  Instructors 
the  class 

maintained  adequate  control  of 

[1] 

[2] 

[3] 

[4] 

[5] 

STUDENTS /TEAMS 

10.  Students  in  other  teams  could  be  seen  clearly 

11.  Students  in  my  own  team  could  be  seen  clearly 

12 .  Students  in  other  teams  could  be  heard  adequately 

13.  Students  in  my  own  team  could  be  heard  adequately 

LEARNING  ACTIVITIES 


14.  Case  studies  provided  a  useful  learning  experience 

[1] 

[2] 

[3] 

[4] 

[5] 

15.  Exercises  provided  a  useful  learning  experience 

[1] 

[2] 

[3] 

[4] 

[5] 

16.  Simulations  provided  a  useful  learning  experience 

[1] 

[2] 

[3] 

[4] 

[5] 

17 .  Written  materials  were  clearly  written 

[1] 

[2] 

[3] 

[4] 

[5] 

[1]  [2]  [3]  [4]  [5] 

[1]  [2]  [3]  [4]  [5] 

[1]  [2]  [3]  [4]  [5] 

[1]  [2]  [3]  [4]  [5] 
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Unsatisfactory — >Out standing 

TRAINING  AIDS 


18.  Training  aids  were  valuable  in  supporting 
instruction 

[1] 

[2] 

[3] 

[4] 

[5] 

19.  Training  aids  were  used  effectively 

[1] 

[2] 

[3] 

[4] 

[5] 

20.  Details  of  training  aids  could  be  clearly  seen 

[1] 

[2] 

[3] 

[4] 

[5] 

INTERACTION/PARTICIPATION 

21.  Interaction  between  instructors  and  students  was 
sufficient  to  support  learning  objectives 

[1] 

[2] 

[3] 

[4] 

[5] 

22.  Interaction  among  the  members  of  my  team  was 
sufficient  to  support  learning  objectives 

[1] 

[2] 

[3] 

[4] 

[5] 

23.  Interaction  among  the  different  teams  was 
sufficient  to  support  learning  objectives 

[1] 

[2] 

[3] 

[4] 

[5] 

24.  Class  participation  was  sufficient  to  support 
learning  objectives 

[1] 

[2] 

[3] 

[4] 

[5] 

OVERALL 

25.  Compare  the  instructor (s)  to  Navy  instructors 
who  have  taught  you  in  the  past 

[1] 

[2] 

[3] 

[4] 

[5] 

26.  Compare  the  course  to  other  Navy  courses  you  have 
taken  in  the  past 

[1] 

[2] 

[3] 

[4] 

[5] 

27.  Rank  how  well  the  course  provided  you  with  skills 
that  can  be  applied  on  the  job. 

[1] 

[2] 

[3] 

[4] 

[5] 

Please  answer  the  following  questions  by  writing  in  your  comments  on  the 
blank  lines . 

28.  What  did  you  like  the  most  about  this  course? _ 


29.  What  did  you  like  the  least  about  this  course? 


30.  Discuss  any  suggestions  you  have  for  improving  the  course 


Appendix  D 
NAVLEAD  Quiz  (Q.3) 
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ANSWER  SHEET  FOR  NAVLEAD  QUIZ  (Q.3) 


1 .  Name^ _ 

Last  First  MI  Rank 

2.  Today's  date _ / _ / _  3.  Location  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


Note:  This  quiz  is  being  used  to  collect  research  data  and 
the  results  will  not  be  recorded  in  your  personal  records. 

Please  answer  the  questions  by  circling  the  letter  of  the 
single  answer  that  is  MOST  CORRECT  on  this  answer  sheet. 


1. 

a 

b 

c 

d 

15.  a 

b 

c 

d 

2. 

a 

b 

c 

d 

16.  a 

b 

c 

d 

3. 

a 

b 

c 

d 

17.  a 

b 

c 

d 

4. 

a 

b 

c 

d 

18.  a 

b 

c 

d 

5, 

a 

b 

c 

d 

19.  a 

b 

c 

d 

6. 

a 

b 

c 

d 

20.  a 

b 

c 

d 

7. 

a 

b 

c 

d 

21.  a 

b 

c 

d 

8. 

a 

b 

c 

d 

22.  a 

b 

c 

d 

9. 

a 

b 

c 

d 

23.  a 

b 

c 

d 

10. 

a 

b 

c 

d 

24 .  a 

b 

c 

d 

11. 

a 

b 

c 

d 

25.  a 

b 

c 

d 

12. 

a 

b 

c 

d 

26.  a 

b 

c 

d 

13. 

a 

b 

c 

d 

27.  a 

b 

c 

d 

14. 

a 

b 

c 

d 

28.  a 

b 

c 

d 

DA 


NAVLEAD  QUIZ  (Q.3) 

DIRECTIONS 

Do  not  turn  this  page  until  the  instructor  tells  you  to  start. 

Mark  the  questions  on  the  answer  sheet.  Do  not  write  in  this  booklet. 


1.  The  Naval  officer  plays  many  different  roles.  Which  of  the  following  statements  is 
NOT  true  about  these  roles? 

a.  Officers  act  as  role  models 

b.  Roles  remain  the  same  throughout  an  officer’s  career 

c.  Roles  are  influenced  by  Navy  "culture" 

d.  Roles  are  influenced  by  command  expectations 

2.  Eliminating  discrimination  and  sexual  harassment  from  the  workplace  is  an  example 
of  which  of  the  following  leadership  principles: 

a.  Knowing  the  job 

b.  Knowing  and  taking  care  of  subordinates 

c.  Directing  and  developing  subordinates 

d.  Promoting  relationships  vital  to  mission  accomplishment 

3.  Giving  unambiguous  taskings  is  an  example  of  which  of  the  following  leadership  prin¬ 
ciples: 

a.  Projecting  a  clear  vision 

b.  Knowing  the  job 

c.  Directing  and  developing  subordinates 

d.  Building  successful  teams 

4.  Which  of  the  following  statements  is  most  correct? 

a.  A  leader  usually  has  a  single  leadership  style 

b.  A  leader  can  change  his/her  leadership  style  at  will 

c.  A  leader  is  seldom  conscious  of  the  leadership  style  s/he  is  using 

d.  A  leader  will  often  have  and  use  a  backup  leadership  style 

5.  The  Coercive  leadership  style  is  most  suited  to  situations  where: 

a.  Subordinates  are  unskilled 

b.  The  task  is  routine 

c.  Subordinates  are  highly  motivated 

d.  Specific  procedures  must  be  followed  exactly 

6.  The  Authoritanan  leadership  style  is  most  suited  to  situations  where: 

a.  There  is  a  crisis  or  emergency 

b.  Subordinates  cannot  have  all  the  information  necessary  for  decision  making 

c.  Performance  measures  and  objectives  are  clear 

d.  Performance  goals  and  standards  are  clear  to  everyone 
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7.  The  Coaching  leadership  style  is  often  used  by  leaders  to: 

a.  Develop  subordinates 

b.  Arrive  at  decisions  by  consensus 

c.  Put  people  before  task  accomplishment 

d.  Provide  a  role  model  to  subordinates 

8.  The  leader  who  relies  on  the  Affiliative  leadership  style  is  unlikely  to: 

a.  Give  task-oriented  feedback 

b.  Show  concern  for  personal  popularity 

c.  Listen  to  subordinates 

d.  Avoid  conflicts 

9.  The  Navy’s  concept  of  Total  Quality  Leadership  focuses  on: 

a.  End  products 

b.  Customers 

c.  Methods 

d.  Inspections 

10.  An  effective  goal  statement  should: 

a.  Specify  the  necessary  action  to  be  taken 

b.  Provide  a  list  of  tasks  and  a  schedule 

c.  Apply  an  action  plan 

d.  Have  at  least  an  80%  chance  for  success 

1 1.  The  first  expectation  of  an  action  planner  is  to: 

a.  Develop  an  action  plan 

b.  Compile  a  list  of  tasks 

c.  Set  goals 

d.  Prepare  a  schedule  for  completion 

12.  Feedback  to  subordinates  is  most  effective  when  it: 

a.  Corrects  an  offensive  behavior 

b.  Is  comprehensive 

c.  Is  evaluative 

d.  Establishes  rapport 


13.  Active  listening  is  best  described  as  a  way  to: 

a.  Provide  emotional  support 

b.  Grasp  meaning  and  feeling 

c.  Understand  thought  processes 

d.  Give  non-threatening  advice 

14.  Power  is  best  described  as  the  ability  to: 

a.  Reward  or  punish  another 

b.  Influence  another 

c.  Exercise  authority  over  another 

d.  Command  the  respect  of  another 

15.  Before  his  Navy  career,  LT  Jones  was  a  star  football  player.  His  subordinates  respect 
and  emulate  him.  This  is  an  example  of: 

a.  Legitimate  power 

b.  Reward  power 

c.  Expert  power 

d.  Identification  power 

16.  LTJG  Smith  was  an  ET  before  receiving  his  commission.  The  ETs  who  work  for  him 
sometimes  come  to  him  for  technical  advice.  This  is  an  example  of: 

a.  Expert  power 

b.  Legitimate  power 

c.  Reward  power 

d.  Identification  power 

17.  Which  of  the  following  is  NOT  a  technique  to  empower  others? 

a.  Paraphrasing  someone’s  idea 

b.  Giving  credit  to  someone  for  an  idea 

c.  Expressing  your  concerns  about  someone’s  idea 

d.  Rejecting  someone’s  idea 

18.  You  attempt  to  support  your  actions  by  using  logical  arguments  that  appeal  to  others’ 
needs  and  interests.  This  is  an  example  of  what  influence  strategy? 

a.  Assertive  persuasion 

b.  Political  coalition  building 

c.  Group  participation 

d.  Common  vision 
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19.  A  commanding  officer  typically  expects  a  prospective  division  officer  to  be: 

a.  Competent  in  the  DO’s  job 

b.  A  skilled  leader 

c.  Dependent  on  the  chief 

d.  Committed  to  learn 

20.  A  department  head  typically  expects  a  division  officer  to: 

a.  Propose  solutions  to  problems 

b.  Teach  the  chief 

c.  Avoid  risks 

d.  Maintain  an  outstanding  division  from  the  first  day  he/she  reports 

21.  The  chief  expects  the  division  officer  to: 

a.  Have  the  solutions  to  problems 

b.  Allow  the  chief  to  lead  the  division 

c.  Prevent  mistakes 

d.  Take  personal  responsibility 

22.  The  Navy  Values  and  Tradition  Project  completed  in  1987  identified  the  following 
core  values: 

a.  Integrity,  tradition,  professionalism 

b.  Honesty,  honor,  responsibility 

c.  Concern  for  people,  patriotism,  courage 

d.  Competence,  teamwork,  loyalty 

23.  Which  of  the  following  guidelines  does  NOT  relate  to  the  leadership  principle  of 
knowing  and  taking  care  of  subordinates? 

a.  Take  time  to  learn  about  subordinates 

b.  Be  sensitive  to  the  effects  of  cultural  and  educational  differences  among  subor¬ 
dinates 

c.  Maintain  good  personal  relations  with  subordinates 

d.  Ensure  that  subordinates’  basic  needs  are  met 
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24.  The  "legal  requirement  or  duty  an  individual  has  been  assigned  to  fulfill"  is  the  for¬ 
mal  definition  of: 

a.  Authority 

b.  Responsibility 

c.  Accountability 

d.  Delegation 

25.  The  preferred  method  of  counseling  to  help  individuals  explore  alternatives  and  take 
initiative  in  solving  a  problem  is: 

a.  Personal  counseling 

b.  Non-directive  counseling 

c.  Directive  counseling 

d.  Active  counseling 

26.  You  are  faced  with  a  situation  in  which  you  must  choose  between  strict  adherence  to 
regulations  or  getting  a  job  done.  This  situation  exemplifies: 

a.  Competing  obligations 

b.  Competing  values 

c.  Ethical  dilemma 

d.  Professional  crisis 

27.  Sexual  harassment  is  a  form  of  sex  discrimination  that  involves  unwelcome  advances, 
requests  for  sexual  favors,  and  other  verbal  or  physical  conduct  of  a  sexual  nature  when: 

a.  Submission  to  or  rejection  of  such  conduct  is  made  either  explicitly  or  implicitly 
a  term  or  condition  of  a  female’s  job,  pay,  or  career 

b.  Submission  to  or  rejection  of  such  conduct  by  a  person  is  used  as  a  basis  for  per¬ 
sonal  decisions  affecting  a  person 

c.  Such  conduct  interferes  with  an  individual’s  right  to  privacy 

d.  Such  conduct  interferes  with  an  individual’s  performance  or  creates  an  intimi¬ 
dating,  hostile,  or  offensive  environment 

28.  As  teams  evolve,  they  typically  go  through  stages.  During  one  of  these  stages, 
members  establish  and  maintain  team  ground  rules  and  boundaries.  This  stage  is: 

a.  Forming 

b.  Storming 

c.  Norming 

d.  Performing 
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Facilitator/Observer  Questionnaire  (Q.4) 


V 
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NAVLEAD  FACILITATOR/OBSERVER  QUESTIONNAIRE  (Q.4) 


1-Name__ _  _  _  2.  Rank 

Last  First  MI 

3.  Your  command _  4.  Your  role:  [  ]  Facilitator 

[  ]  Observer 
[]  Other _ 

5.  Location  (check  one):  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


DIRECTIONS 

1.  Complete  one  of  the  attached  rating  forms  for  EACH  DAY,  Monday  through  Friday, 

2.  Rate  each  of  the  course  dimensions  by  putting  a  checkmark  in  the  box  that  is  most 
appropriate.  The  scale  is  1  (unsatisfactoiy)  through  5  (outstanding),  with  3  being 
average. 

3.  Leave  items  that  do  not  apply  blank. 

4.  If  you  are  rating  a  VTT  class,  the  "local"  classroom  is  the  site  where  the  instructor  is 
located.  "Remote"  classrooms  are  the  other  sites. 

5.  If  you  are  rating  a  traditional  (non  VTT)  class,  ignore  statements  relating  to  "remote" 
classrooms  and  make  ratings  for  "local"  classrooms  only. 


Check  day:  [  ]  Monday  [  ]  Tuesday  [  ]  Wednesday  [  ]  Thursday  [  ]  Friday 


Unsatisfactory — >Out standing 

1.  Effectiveness  of  instructor's  presentations  [1]  [2]  [3]  [4]  [5] 

2.  Effectiveness  of  case  studies  [1]  [2]  [3]  [4]  [5] 

3.  Effectiveness  of  exercises  [1]  [2]  [3]  [4]  [5] 

4.  Effectiveness  of  simulations  [1]  [2]  [3]  [4]  [5] 

5.  Degree  of  instructor  control 

a.  Local  classroom  [1]  [2]  [3]  [4]  [5] 

b.  Remote  classrooms  [1]  [2]  [3]  [4]  [5] 

6.  Difficulty  of  conducting  instruction  [1]  [2]  [3]  [4]  [5] 

7.  Amount  of  instructor-student  interaction  [1]  [2]  [3]  [4]  [5] 

8.  Success  in  meeting  learning  objectives  [1]  [2]  [3]  [4]  [5] 

9 .  Amount  of  student  participation 

a.  Local  classroom  [1]  [2]  [3]  [4]  [5] 

b.  Remote  classrooms  [1]  [2]  [3]  [4]  [5] 

10.  Amount  of  interaction  among  teams  [1]  [2]  [3]  [4]  [5] 

11.  Degree  of  team  cohesiveness 

a.  Local  classroom  [1]  [2]  [3]  [4]  [5] 

b.  Remote  classrooms  [1]  [2]  [3]  [4]  [5] 

12.  Amount  of  student-student  interaction 

a.  Local  classroom  [1]  [2]  [3]  [4]  [5] 

b.  Remote  classrooms  [1]  [2]  [3]  [4]  [5] 

13.  Amount  of  student-student  interaction 

(between  local  and  remote  classes)  [1]  [2]  [3]  [4]  [5] 

14.  Quality  of  student  presentations 

a.  Local  classroom  [1]  [2]  [3]  [4]  [5] 

b.  Remote  classrooms  [1]  [2]  [3]  [4]  [5] 

Comments : 


Appendix  F 

Performance  Activities  (Q.5) 


LESSON  3.5  PRESENTATION  CRITIQUE  (Q.5) 


1-Name_ _  _  _  2.  Rank 

Last  First  MI 

3.  Your  command _  4.  Your  role:  [  ]  Facilitator 

[  ]  Observer 
[]  Other _ 


5.  Your  location:  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


DIRECTIONS 

1.  Complete  the  attached  rating  forms  on  day  3  for  the  student  presentations  made  during 
lesson  3.5  (DIVO/CPO  relationships) 

2.  Rate  each  presentation  as  it  is  made  on  one  of  the  attached  scoring  sheets 

3.  Rate  ONLY  the  first  five  presentations 

4.  For  each  statement  (1  through  9),  check  the  appropriate  box  corresponding  to  a  scale 
of  1  (unsatisfactory)  through  5  (outstanding),  with  3  being  average. 
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LESSON  3.5  PRESENTATION  CRITIQUE 


Presentation  number:  [1]  [2]  [3]  [4]  [5] 

Originates  from:  [  ]  Charleston 

(  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


Unsatisfactory — >Out standing 


1.  strategy  is  specific 

[1] 

[2] 

[3] 

[4] 

[5] 

2.  Strategy  is  realistic 

[1] 

[2] 

[3] 

[4] 

[5] 

3.  Each  need  identified  by  class  is  addressed 

[1] 

[2] 

[3] 

[4] 

[5] 

4  -  Power  bases  and  influence  behaviors  will 
be  effective 

[1] 

[2] 

[3] 

[4] 

[5] 

5.  Influence  strategies  will  work  up  the  chain 
of  command 

[1] 

[2] 

[3] 

[4] 

[5] 

6.  Influence  strategies  will  work  down  the  chain 
of  command 

[1] 

[2] 

[3] 

[4] 

[5] 

7 .  Presentation  was  delivered  within  allotted  time 

[1] 

[2] 

[3] 

[4] 

[5] 

8.  Presentation  was  confident  and  professional 

[1] 

[2] 

[3] 

[4] 

[5] 

9.  Speaker  responded  appropriately  to  questions 

[1] 

[2] 

[3] 

[4] 

[5] 
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LESSON  5.4  PRESENTATION  CRITIQUE  (Q.5) 


LName  _  _  _  2.  Rank 

Last  First  MI 

3.  Your  command _  4.  Your  role:  [  ]  Facilitator 

[  ]  Observer 
[  ]  Other _ 


5.  Your  location:  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


DIRECTIONS 

1.  Complete  the  attached  rating  forms  on  day  5  for  the  student  presentations  made  during 
lesson  5.4  (assessing  the  division) 

2.  Rate  each  presentation  as  it  is  made  on  one  of  the  attached  scoring  sheets 

3.  Rate  ONLY  the  first  five  presentations 

4.  For  each  statement  (1  through  15),  check  the  appropriate  box  corresponding  to  a  scale 
of  1  (unsatisfactory)  through  5  (outstanding),  with  3  being  average. 
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LESSON  5.4  PRESENTATION  CRITIQUE 


Presentation  number:  [1]  [2]  [3]  [4]  [5] 

Originates  from:  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


Unsatisfactory — >Out standing 

FIRST  SET  OF  QUESTIONS  (New  DO) 


1.  Questions  assess  relationships  among  individuals 


in  division 

[1] 

[2] 

[3] 

[4] 

[5] 

2.  Questions  assess  relationships  among  petty  officers 
and  subordinates 

[1] 

[2] 

[3] 

[4] 

[5] 

3.  Questions  assess  relationships  among  chief 
and  petty  officers 

[1] 

[2] 

[3] 

[4] 

[5] 

4.  Questions  assess  relationship  among  division  officer 
and  chief 

[1] 

[2] 

[3] 

[4] 

[5] 

5.  Questions  assess  division  performance  as  a  unit 

[1] 

[2] 

[3] 

[4] 

[5] 

6.  Questions  identify  division  strengths  as  a  unit 

[1] 

[2] 

[3] 

[4] 

[5] 

7.  Questions  identify  division  weaknesses  as  a  unit 

[1] 

[2] 

[3] 

[4] 

[5] 

8.  Questions  assess  division  cohesiveness  as  a  unit 

[1] 

[2] 

[3] 

[4] 

[5] 

9 .  Questions  provide  information  needed  for  team 
building 

[1] 

[2] 

[3] 

[4] 

[5] 

Unsatisfactory — >Out standing 

SECOND  SET  OF  QUESTIONS  (2-3  wks  later) 


10.  Questions  assess  relationships  among  individuals 


in  division 

[1] 

[2] 

[3] 

[4] 

[5] 

11.  Questions  assess  relationships  among 
petty  officers  and  subordinates 

[1] 

[2] 

[3] 

[4] 

[5] 

12.  Questions  assess  relationships  among  chief 
and  petty  officers 

[1] 

[2] 

[3] 

[4] 

[5] 

13.  Questions  assess  relationship  among  division 
officer  and  chief 

[1] 

[2] 

[3] 

[4] 

[5] 

14.  Questions  assess  division  performance  as  a  unit 

[1] 

[2] 

[3] 

[4] 

[5] 

15.  Questions  assess  division  cohesiveness  as  a  unit 

[1] 

[2] 

[3] 

[4] 

[5] 
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Appendix  G 

Class  Participation  Tally  (Q.6) 
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CLASS  PARTICIPATION  TALLY  SHEET  (Q.6) 


l-Name _  _  _  2.  Rank 

Last  First  MI 

3.  Your  command _ _ _  4.  Your  role:  [  ]  Facilitator 

[  ]  Observer 
[  ]  Other _ 


5.  Location  (check  one):  [  ]  Charleston 

[  ]  Damneck 
[  ]  Newport 
[  ]  Little  Creek 


DIRECTIONS 


1.  Complete  a  separate  block  of  the  attached  tally  sheet  for  each  day,  Monday  through 
Friday. 


2.  Log  data  for  30-minute  intervals  during  the  time  periods  0900-1000  and  1300-1400. 

3.  Make  a  /  for  each  student  question  or  comment  in  the  appropriate  cell.  To  rate  a  /  the 
remark  must: 

a.  be  directed  at  instructors  or  students, 

b.  relate  to  course  content,  and 

c.  be  intended  for  the  class  to  hear 

4.  If  you  are  observing  a  WTT  class,  log  data  ONLY  for  the  tables  in  the  classroom  you 
are  observing  directly.  (For  example,  if  you  are  at  Charleston,  log  data  for  Charleston 
but  not  for  Damneck  or  Newport.) 
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Time  Team  1 
Interval 


Team  2 


Team  3 


Team  4 


Team  5 


0900-+- 

I 

0930-+- 

I 

1000-+- 

1300-+- 

I 

1330-+- 

I 

1400-+- 

0900-+- 

I 

0930-+- 

I 

1000-+- 

1300-+- 

I 

1330-+- 

I 

1400-+- 

0900-+- 

I 

0930-+- 

I 

1000-+- 

1300-+- 

I 

1330-+- 

I 

1400-+— 

0900-+— 

I 

0930-+— 

I 

1000-+-- 

1300-+— 

I 

1330-+— 

I 

1400-+-- 

0900-+-- 

I 

0930-+ — 

I 

1000-+ — 
1300-+ — 

I 

1330-+ - 

I 

1400-+ - 


MONDAY 


TUESDAY 


WEDNESDAY 


THURSDAY 


FRIDAY 
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Distribution  List 


Distribution: 

Chief  of  Naval  Education  and  Training  (Code  OOQ) 

Chief  of  Naval  Personnel  (PERS-62),  (PERS-OOH) 

Defense  Technical  Infonnation  Center  (DTIC)  (4) 

Copy  to: 

Commanding  Officer,  Fleet  Training  Center,  San  Diego  (Code  021.2,  VTT) 
Fleet  Combat  Training  Center,  Atlantic  (Code  N713,  VTT) 

Director,  Naval  Leader  Training  Unit,  Little  Creek 
Director,  Naval  Leader  Training  Unit,  Coronado 


